{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "390311fb-1b92-4be6-b535-dbdbea92c2eb",
   "metadata": {},
   "source": [
    "# 1.K近邻算法"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "163f3055-f805-4862-930a-d10e3c50270a",
   "metadata": {},
   "source": [
    "## 1.1 基本概念"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "33ba0834-b147-4619-b582-c8259827f891",
   "metadata": {},
   "source": [
    "K 近邻（K-Nearest Neighbors，简称 KNN）是一种简单而强大的**监督学习算法**，可用于**分类**和**回归**任务。\n",
    "\n",
    "它的核心思想是：一个样本与数据集中的 **k 个最相似（最近邻）** 的样本中的大多数属于同一类别，则该样本也属于这个类别。\n",
    "\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250822101038282.png\" alt=\"image-20250822101038282\" style=\"zoom: 33%;\" />\n",
    "\n",
    "- 如图所示，已知数据集中有八个样本\n",
    "- 每个样本具有两个特征值（X轴，Y轴），标签为红色和蓝色\n",
    "- 新样本为绿色的点\n",
    "- 距离绿色点最近的 $k=3$ 个点为蓝色，则新样本标签为蓝色\n",
    "\n",
    "**关于距离的问题**\n",
    "\n",
    "目前K近邻讨论的距离为 **欧拉距离**：\n",
    "\n",
    "- 在二维维度中：\n",
    "$\\sqrt{(x^{(a)}-x^{(b)})^2+(y^{(a)}-y^{(b)})^2}$\n",
    "\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250822103206058.png\" alt=\"image-20250822103206058\" style=\"zoom:33%;center:middle\" />\n",
    "\n",
    "- 在三维维度中：\n",
    "$\\sqrt{(x^{(a)}-x^{(b)})^2+(y^{(a)}-y^{(b)})^2+(z^{(a)}-z^{(b)})^2}$\n",
    "\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250822104728365.png\" alt=\"image-20250822104728365\" style=\"zoom:33%;\" />\n",
    "\n",
    "- 在N维维度中：\n",
    "$\\sqrt{(X^{(a)}_1-X^{(b)}_1)^2+(X^{(a)}_2-X^{(b)}_2)^2+...+(X^{(a)}_n-X^{(b)}_n)^2}$\n",
    "\n",
    "- 将上述公式综合为：\n",
    "$$\n",
    "\\sqrt{\\sum^n_{i=1}{(X^{(a)}_i-X^{(b)}_i)^2}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9cef088-24c7-49f8-8f0d-7058e968e30b",
   "metadata": {},
   "source": [
    "## 1.2 基本实现"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bc73796-e57b-4919-99b8-f13c2f5ed7e8",
   "metadata": {},
   "source": [
    "**（1）导入所需要的包和数据**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ae17b164-fddf-43b3-9b09-454ae8c9353b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 原始数据集：特征矩阵 X\n",
    "# 10个样本 每个样本有2个特征\n",
    "raw_data_X = [[3.393533211, 2.331273381],\n",
    "              [3.110073483, 1.781539638],\n",
    "              [1.343808831, 3.368360954],\n",
    "              [3.582294042, 4.679179110],\n",
    "              [2.280362439, 2.866990263],\n",
    "              [7.423436942, 4.696522875],\n",
    "              [5.745051997, 3.533989803],\n",
    "              [9.172168622, 2.511101045],\n",
    "              [7.792783481, 3.424088941],\n",
    "              [7.939820817, 0.791637231]\n",
    "]\n",
    "\n",
    "# 原始数据集：标签向量 y\n",
    "raw_data_y = [0,0,0,0,0,1,1,1,1,1]\n",
    "\n",
    "# 导入Python关于矩阵运算的包 numpy\n",
    "import numpy as np\n",
    "\n",
    "# 将原始数据封装为np数据（线性代数中的矩阵）\n",
    "X_train = np.array(raw_data_X) # 将Python的二维列表 转换为 矩阵\n",
    "y_train = np.array(raw_data_y) # 将Python的一维列表 转换为 向量\n",
    "\n",
    "# 待预测的数据\n",
    "x = np.array([8.09, 3.36])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "647f10a1-dad8-4fe6-92b9-19fd6e1fb474",
   "metadata": {},
   "source": [
    "**（2）画图显示数据间关系**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "482c8132-e5a9-443c-8a61-319f18ddce8b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAJxhJREFUeJzt3X9wFOd9x/HPcsKHinVno1rixJ2MiF3AYsAMMEUOolBhXEQ1eARx2zgxgaQdtTJI1tChwp1JU8eV27odwTiFyPEvzBjSQYLgYmPcFgl5BjfIFg6xMcE1sWT5ZOIZ+06Q9mQd2z+uuvgsCXTSSY/u7v2a2SH77LPsdycJ+7ndZ5+1bNu2BQAAYMgk0wUAAID0RhgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYFSG6QKG4+rVq/roo4+UlZUly7JMlwMAAIbBtm319PQoLy9PkyYNff8jKcLIRx99JJ/PZ7oMAAAwAp2dnfJ6vUNuT4owkpWVJSlyMi6Xy3A1AABgOILBoHw+X/Q6PpSkCCP9j2ZcLhdhBACAJHO9IRYMYAUAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYlRSTngEAUkM4LLW2Sn6/5PFIxcWSw2G6KphGGAEAjIumJqmqSvrww9+0eb3Szp1Sebm5umAej2mQvsJhqblZ2r8/8mc4bLoiIGU1NUkbNsQGEUnq6oq0NzWZqQsTA2EE6ampSZo5U1q5Uvr61yN/zpzJv4jAGAiHI3dEbHvgtv626mp+D6QzwgjSDz/RgHHV2jrw/25fZNtSZ2ekH9ITYQTphZ9owLjz+xPbD6mHMIL0wk80YNx5PInth9RDGEF64ScaMO6KiyNvzVjW4NstS/L5Iv2QnggjSC/8RAPGncMReX1XGhhI+tfr65lvJJ0RRpBe+IkGGFFeLh08KM2YEdvu9UbamWckvTHpGdJL/0+0DRsiweOLA1n5iQaMqfJyad06ZmDFQIQRpJ/+n2iDTQVZX89PNGAMORzSihWmq8BEQxhBeuInGgBMGIQRpC9+ogHAhMAAVgAAYBRhBAAAGEUYAQAARhFGAACAUaMKI3V1dbIsS9XV1UP2aW5ulmVZA5Z33313NIcGAAApYsRv05w+fVoNDQ2aP3/+sPqfP39eLpcrun7LLbeM9NAAACCFjOjOyOXLl3X//ffrySef1M033zysfXJycjR9+vTo4mA+BwAAoBGGkcrKSq1du1arVq0a9j4LFy6Ux+NRSUmJTpw4cc2+oVBIwWAwZgEAAKkp7sc0Bw4c0JtvvqnTp08Pq7/H41FDQ4MWLVqkUCik559/XiUlJWpubtby5csH3aeurk7f+9734i0NAAAkIcu2v/ilsGvr7OzU4sWLdfz4cS1YsECStGLFCt15552qr68f9kHLyspkWZaOHDky6PZQKKRQKBRdDwaD8vl8CgQCMeNOAADAxBUMBuV2u697/Y7rMc0bb7yhS5cuadGiRcrIyFBGRoZaWlq0a9cuZWRkKBwOD+vvWbp0qS5cuDDkdqfTKZfLFbMAAIDUFNdjmpKSEp09ezambdOmTZozZ462b98+7EGp7e3t8ng88RwaAACkqLjCSFZWlubNmxfTNnXqVGVnZ0fba2tr1dXVpb1790qS6uvrNXPmTBUWFqq3t1f79u1TY2OjGhsbE3QKAAAgmSX8q71+v18dHR3R9d7eXm3btk1dXV3KzMxUYWGhjh49qtLS0kQfGgAAJKG4BrCaMtwBMAAAYOIYkwGsAAAAiUYYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGDUqMJIXV2dLMtSdXX1Nfu1tLRo0aJFmjJlimbNmqU9e/aM5rAAACCFjDiMnD59Wg0NDZo/f/41+128eFGlpaUqLi5We3u7duzYoa1bt6qxsXGkhwYAAClkRGHk8uXLuv/++/Xkk0/q5ptvvmbfPXv2KD8/X/X19Zo7d66+853vaPPmzXr88cdHVDAAAEgtIwojlZWVWrt2rVatWnXdvqdOndLq1atj2u655x61tbXp888/H3SfUCikYDAYswAAgNQUdxg5cOCA3nzzTdXV1Q2rf3d3t3Jzc2PacnNz1dfXp08++WTQferq6uR2u6OLz+eLt0wAAJAk4gojnZ2dqqqq0r59+zRlypRh72dZVsy6bduDtverra1VIBCILp2dnfGUCQAAkkhGPJ3feOMNXbp0SYsWLYq2hcNhnTx5Uk888YRCoZAcDkfMPtOnT1d3d3dM26VLl5SRkaHs7OxBj+N0OuV0OuMpDQAAJKm4wkhJSYnOnj0b07Zp0ybNmTNH27dvHxBEJKmoqEgvvvhiTNvx48e1ePFiTZ48eQQlAwCAVBJXGMnKytK8efNi2qZOnars7Oxoe21trbq6urR3715JUkVFhZ544gnV1NToT//0T3Xq1Ck99dRT2r9/f4JOAQAAJLOEz8Dq9/vV0dERXS8oKNBLL72k5uZm3XnnnXrkkUe0a9curV+/PtGHBgAASciy+0eTTmDBYFBut1uBQEAul8t0OQAAYBiGe/3m2zQAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwKgM0wUAwHgJh6XWVsnvlzweqbhYcjhMVwWAMAIgLTQ1SVVV0ocf/qbN65V27pTKy83VBYDHNADSQFOTtGFDbBCRpK6uSHtTk5m6AETEFUZ2796t+fPny+VyyeVyqaioSC+//PKQ/Zubm2VZ1oDl3XffHXXhADAc4XDkjohtD9zW31ZdHekHwIy4HtN4vV499thjuu222yRJzz33nNatW6f29nYVFhYOud/58+flcrmi67fccssIywWA+LS2Drwj8kW2LXV2RvqtWDFuZQH4grjCSFlZWcz6o48+qt27d+v111+/ZhjJycnRTTfdNKICAWA0/P7E9gOQeCMeMxIOh3XgwAFduXJFRUVF1+y7cOFCeTwelZSU6MSJE9f9u0OhkILBYMwCACPh8SS2H4DEizuMnD17VjfeeKOcTqcqKip06NAh3XHHHYP29Xg8amhoUGNjo5qamjR79myVlJTo5MmT1zxGXV2d3G53dPH5fPGWCQCSIq/ver2SZQ2+3bIkny/SD4AZlm0PNqxraL29vero6NBnn32mxsZG/ehHP1JLS8uQgeTLysrKZFmWjhw5MmSfUCikUCgUXQ8Gg/L5fAoEAjFjTwBgOPrfppFiB7L2B5SDB3m9FxgLwWBQbrf7utfvuO+M3HDDDbrtttu0ePFi1dXVacGCBdq5c+ew91+6dKkuXLhwzT5OpzP6xk7/AgAjVV4eCRwzZsS2e70EkVQQDkvNzdL+/ZE/eTMq+Yx60jPbtmPuYlxPe3u7PDycBTDOysuldeuYgTXVMJldaogrjOzYsUNr1qyRz+dTT0+PDhw4oObmZh07dkySVFtbq66uLu3du1eSVF9fr5kzZ6qwsFC9vb3at2+fGhsb1djYmPgzAYDrcDh4fTeV9D9++/Jgg/7J7LjrlTziCiMff/yxvvnNb8rv98vtdmv+/Pk6duyY7r77bkmS3+9XR0dHtH9vb6+2bdumrq4uZWZmqrCwUEePHlVpaWlizwIAkFauN5mdZUUms1u3jrtfySDuAawmDHcADAAgPTQ3SytXXr/fiRPcDTNpuNfv9P1QHp/vBICklejJ7MJXw2rtaJW/xy9PlkfF+cVyTOKaMF7SM4ww4gkAkloiJ7NrOtekqmNV+jD4m2uC1+XVzj/YqfK5XBPGQ/p9tZfPdwJA0kvUZHZN55q04V83xAQRSeoKdmnDv25Q0zmuCeMhvcIIn+8EgJTgcERuZksDA0n/en39tZ++h6+GVXWsSrYGXhP626qPVSt8lWvCWEuvMBLP5zsBABPaaCeza+1oHXBH5Its2eoMdqq1g2vCWEuvMSN8vhMAUspoJrPz9wzv3/rh9sPIpVcY4fOdAJByRjqZnSdreP/WD7cfRi69HtPw+U4AwP8rzi+W1+WVpcGvCZYs+Vw+FedzTRhr6RVGEjHiCQCQEhyTHNr5B5FrwpcDSf96/R/UM9/IOEivMCLx+U4AQFT53HIdvO+gZrhirwlel1cH7zvIPCPjJH2ng2cGVgDA/2MG1rHBdPDXw+c7AQD/zzHJoRUzV5guI22l32MaAAAwoRBGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABgVVxjZvXu35s+fL5fLJZfLpaKiIr388svX3KelpUWLFi3SlClTNGvWLO3Zs2dUBQMAgNQSVxjxer167LHH1NbWpra2Nv3+7/++1q1bp7fffnvQ/hcvXlRpaamKi4vV3t6uHTt2aOvWrWpsbExI8QAAIPlZtm3bo/kLpk2bpn/8x3/Ut7/97QHbtm/friNHjujcuXPRtoqKCr311ls6derUsI8RDAbldrsVCATkcrlGUy4AABgnw71+j3jMSDgc1oEDB3TlyhUVFRUN2ufUqVNavXp1TNs999yjtrY2ff7550P+3aFQSMFgMGYBAACpKe4wcvbsWd14441yOp2qqKjQoUOHdMcddwzat7u7W7m5uTFtubm56uvr0yeffDLkMerq6uR2u6OLz+eLt0wAAJAk4g4js2fP1pkzZ/T666/rz//8z7Vx40a98847Q/a3LCtmvf+p0Jfbv6i2tlaBQCC6dHZ2xlsmAABIEhnx7nDDDTfotttukyQtXrxYp0+f1s6dO/XDH/5wQN/p06eru7s7pu3SpUvKyMhQdnb2kMdwOp1yOp3xlgYAAJLQqOcZsW1boVBo0G1FRUV69dVXY9qOHz+uxYsXa/LkyaM9NAAASAFxhZEdO3aotbVVv/zlL3X27Fk9/PDDam5u1v333y8p8njlgQceiPavqKjQBx98oJqaGp07d05PP/20nnrqKW3bti2xZwEAAJJWXI9pPv74Y33zm9+U3++X2+3W/PnzdezYMd19992SJL/fr46Ojmj/goICvfTSS3rooYf0gx/8QHl5edq1a5fWr1+f2LMAAABJa9TzjIwH5hkBACD5DPf6HfcAVkxw4bDU2ir5/ZLHIxUXSw6H6aoAABgSYSSVNDVJVVXShx/+ps3rlXbulMrLzdUFAMA18NXeVNHUJG3YEBtEJKmrK9Le1GSmLgAAroMwkgrC4cgdkcGG//S3VVdH+gEAMMEQRlJBa+vAOyJfZNtSZ2ekHwAAEwxhJBX4/YntBwDAOCKMpAKPJ7H9AAAYR4SRVFBcHHlrZqiPD1qW5PNF+gEAMMEQRlKBwxF5fVcaGEj61+vrmW8EADAhEUZSRXm5dPCgNGNGbLvXG2lnnhEAwATFpGeppLxcWreOGVgBAEmFMJJqHA5pxQrTVQAAMGw8pgEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgVIbpAgAAgBnhsNTaKvn9kscjFRdLDsf41xHXnZG6ujotWbJEWVlZysnJ0b333qvz589fc5/m5mZZljVgeffdd0dVOAAAGLmmJmnmTGnlSunrX4/8OXNmpH28xRVGWlpaVFlZqddff12vvvqq+vr6tHr1al25cuW6+54/f15+vz+63H777SMuGgAAjFxTk7Rhg/Thh7HtXV2R9vEOJJZt2/ZId/7Vr36lnJwctbS0aPny5YP2aW5u1sqVK/Xpp5/qpptuGtFxgsGg3G63AoGAXC7XSMsFACDthcOROyBfDiL9LEvyeqWLF0f/yGa41+9RDWANBAKSpGnTpl2378KFC+XxeFRSUqITJ05cs28oFFIwGIxZAADA6LW2Dh1EJMm2pc7OSL/xMuIwYtu2ampqtGzZMs2bN2/Ifh6PRw0NDWpsbFRTU5Nmz56tkpISnTx5csh96urq5Ha7o4vP5xtpmQAA4Av8/sT2S4QRP6aprKzU0aNH9dprr8nr9ca1b1lZmSzL0pEjRwbdHgqFFAqFouvBYFA+n4/HNAAAjFJzc2Sw6vWcOCGtWDG6Y43pY5otW7boyJEjOnHiRNxBRJKWLl2qCxcuDLnd6XTK5XLFLAAAYPSKiyNjQixr8O2WJfl8kX7jJa4wYtu2HnzwQTU1Nek///M/VVBQMKKDtre3y+PxjGhfAAAwcg6HtHNn5D9/OZD0r9fXj+98I3FNelZZWakXXnhBP/nJT5SVlaXu7m5JktvtVmZmpiSptrZWXV1d2rt3rySpvr5eM2fOVGFhoXp7e7Vv3z41NjaqsbExwacCAACGo7xcOnhQqqqKHczq9UaCSHn5+NYTVxjZvXu3JGnFlx4iPfPMM/rWt74lSfL7/ero6Ihu6+3t1bZt29TV1aXMzEwVFhbq6NGjKi0tHV3lAABgxMrLpXXrJsYMrKOaZ2S8MM8IAADJZ1zmGQEAABgtwggAADCKMAIAAIyKawArMOFNlO9hAwCGjTCC1NHUNPh7ajt3jv97agCAYeMxDVLDRPseNgBg2AgjSH7hcOSOyGBvqfe3VVdH+gEAJhzCCJLfRPweNgBg2AgjSH4T8XvYAIBhI4wg+Q33o4t8nBEAJiTCCJLfRPweNgBg2AgjSH4T8XvYAIBhI4wgNfR/D3vGjNh2rzfSzjwjADBhMekZUsdE+h42AGDYCCNILQ6HtGKF6SoAAHHgMQ0AADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMCquMFJXV6clS5YoKytLOTk5uvfee3X+/Pnr7tfS0qJFixZpypQpmjVrlvbs2TPiggEAQGqJK4y0tLSosrJSr7/+ul599VX19fVp9erVunLlypD7XLx4UaWlpSouLlZ7e7t27NihrVu3qrGxcdTFAwCA5GfZtm2PdOdf/epXysnJUUtLi5YvXz5on+3bt+vIkSM6d+5ctK2iokJvvfWWTp06NazjBINBud1uBQIBuVyukZYLAADG0XCv36MaMxIIBCRJ06ZNG7LPqVOntHr16pi2e+65R21tbfr8889Hc3gAAJACMka6o23bqqmp0bJlyzRv3rwh+3V3dys3NzemLTc3V319ffrkk0/k8XgG7BMKhRQKhaLrwWBwpGUCAIAJbsR3Rh588EH97Gc/0/79+6/b17KsmPX+J0Nfbu9XV1cnt9sdXXw+30jLBAAAE9yIwsiWLVt05MgRnThxQl6v95p9p0+fru7u7pi2S5cuKSMjQ9nZ2YPuU1tbq0AgEF06OztHUiYAAEgCcT2msW1bW7Zs0aFDh9Tc3KyCgoLr7lNUVKQXX3wxpu348eNavHixJk+ePOg+TqdTTqczntIAAECSiuvOSGVlpfbt26cXXnhBWVlZ6u7uVnd3t/7nf/4n2qe2tlYPPPBAdL2iokIffPCBampqdO7cOT399NN66qmntG3btsSdBQAASFpxhZHdu3crEAhoxYoV8ng80eXHP/5xtI/f71dHR0d0vaCgQC+99JKam5t155136pFHHtGuXbu0fv36xJ0FAABIWqOaZ2S8MM8IAADJZ1zmGQEAABgtwggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjBrxV3uBMREOS62tkt8veTxScbHkcJiuCgAwhggjmDiamqSqKunDD3/T5vVKO3dK5eXm6gIAjCke02BiaGqSNmyIDSKS1NUVaW9qMlMXAGDMEUZgXjgcuSMy2JcJ+tuqqyP9AAAphzAC81pbB94R+SLbljo7I/0AACmHMALz/P7E9gMAJBXCCMzzeBLbDwCQVAgjMK+4OPLWjGUNvt2yJJ8v0g8AkHIIIzDP4Yi8visNDCT96/X1zDcCACmKMIKJobxcOnhQmjEjtt3rjbQzzwgApCwmPcPEUV4urVvHDKwAkGYII5hYHA5pxQrTVQAAxhGPaQAAgFGEEQAAYBRhBAAAGEUYAQAARhFGAACAUYQRAABgFGEEAAAYRRgBAABGEUYAAIBRhBEAAGAUYQQAABhFGAEAAEYRRgAAgFFxh5GTJ0+qrKxMeXl5sixLhw8fvmb/5uZmWZY1YHn33XdHWjMAAEghGfHucOXKFS1YsECbNm3S+vXrh73f+fPn5XK5ouu33HJLvIcGAAApKO4wsmbNGq1ZsybuA+Xk5Oimm26Kez8AAJDaxm3MyMKFC+XxeFRSUqITJ06M12EBAMAEF/edkXh5PB41NDRo0aJFCoVCev7551VSUqLm5mYtX7580H1CoZBCoVB0PRgMjnWZAADAkDEPI7Nnz9bs2bOj60VFRers7NTjjz8+ZBipq6vT9773vbEuDQAATABGXu1dunSpLly4MOT22tpaBQKB6NLZ2TmO1QEAgPE05ndGBtPe3i6PxzPkdqfTKafTOY4VAQAAU+IOI5cvX9Z7770XXb948aLOnDmjadOmKT8/X7W1terq6tLevXslSfX19Zo5c6YKCwvV29urffv2qbGxUY2NjYk7CwAAkLTiDiNtbW1auXJldL2mpkaStHHjRj377LPy+/3q6OiIbu/t7dW2bdvU1dWlzMxMFRYW6ujRoyotLU1A+QAAINlZtm3bpou4nmAwKLfbrUAgEDNxGgAAmLiGe/3m2zQAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAo+IOIydPnlRZWZny8vJkWZYOHz583X1aWlq0aNEiTZkyRbNmzdKePXtGUisAAEhBcYeRK1euaMGCBXriiSeG1f/ixYsqLS1VcXGx2tvbtWPHDm3dulWNjY1xFwsAAFJPRrw7rFmzRmvWrBl2/z179ig/P1/19fWSpLlz56qtrU2PP/641q9fH+/hAQBAihnzMSOnTp3S6tWrY9ruuecetbW16fPPPx/rwwMAgAku7jsj8eru7lZubm5MW25urvr6+vTJJ5/I4/EM2CcUCikUCkXXg8HgWJcJAAAMGZe3aSzLilm3bXvQ9n51dXVyu93RxefzjXmNAADAjDEPI9OnT1d3d3dM26VLl5SRkaHs7OxB96mtrVUgEIgunZ2dY10mAAAwZMwf0xQVFenFF1+MaTt+/LgWL16syZMnD7qP0+mU0+kc69IAAMAEEPedkcuXL+vMmTM6c+aMpMiru2fOnFFHR4ekyF2NBx54INq/oqJCH3zwgWpqanTu3Dk9/fTTeuqpp7Rt27bEnAEAAEhqcd8ZaWtr08qVK6PrNTU1kqSNGzfq2Wefld/vjwYTSSooKNBLL72khx56SD/4wQ+Ul5enXbt28VovAACQJFl2/2jSCSwYDMrtdisQCMjlcpkuBwAADMNwr998mwYAABhFGAEAAEYRRgAAgFGEEQAAYBRhBAAAGEUYAQAARo35DKwAAJgUDkutrZLfL3k8UnGx5HCYrgpfRBgBAKSspiapqkr68MPftHm90s6dUnm5uboQi8c0AICU1NQkbdgQG0Qkqasr0t7UZKYuDEQYAQCknHA4ckdksDnG+9uqqyP9YB5hBACQclpbB94R+SLbljo7I/1gHmEEAJBy/P7E9sPYIowAAFKOx5PYfhhbhBEAQMopLo68NWNZg2+3LMnni/SDeYQRAEDKcTgir+9KAwNJ/3p9PfONTBSEEQBASiovlw4elGbMiG33eiPtzDMycTDpGQAgZZWXS+vWMQPrREcYAQCkNIdDWrHCdBW4Fh7TAAAAowgjAADAKMIIAAAwijACAACMIowAAACjCCMAAMAowggAADCKMAIAAIwijAAAAKOSYgZW27YlScFg0HAlAABguPqv2/3X8aEkRRjp6emRJPl8PsOVAACAePX09Mjtdg+53bKvF1cmgKtXr+qjjz5SVlaWenp65PP51NnZKZfLZbq0MREMBlP+HKX0OM90OEeJ80wl6XCOUnqc50Q4R9u21dPTo7y8PE2aNPTIkKS4MzJp0iR5vV5JkmVZkiSXy5Wy/wPqlw7nKKXHeabDOUqcZypJh3OU0uM8TZ/jte6I9GMAKwAAMIowAgAAjEq6MOJ0OvXd735XTqfTdCljJh3OUUqP80yHc5Q4z1SSDucopcd5JtM5JsUAVgAAkLqS7s4IAABILYQRAABgFGEEAAAYRRgBAABGJU0YOXnypMrKypSXlyfLsnT48GHTJSVcXV2dlixZoqysLOXk5Ojee+/V+fPnTZeVULt379b8+fOjk/AUFRXp5ZdfNl3WmKurq5NlWaqurjZdSkL9zd/8jSzLilmmT59uuqyE6+rq0je+8Q1lZ2frt37rt3TnnXfqjTfeMF1WQs2cOXPAf5eWZamystJ0aQnT19env/7rv1ZBQYEyMzM1a9Ys/e3f/q2uXr1qurSE6+npUXV1tW699VZlZmbqrrvu0unTp02XNaSkmIFVkq5cuaIFCxZo06ZNWr9+velyxkRLS4sqKyu1ZMkS9fX16eGHH9bq1av1zjvvaOrUqabLSwiv16vHHntMt912myTpueee07p169Te3q7CwkLD1Y2N06dPq6GhQfPnzzddypgoLCzUv//7v0fXHQ6HwWoS79NPP9VXv/pVrVy5Ui+//LJycnL03//937rppptMl5ZQp0+fVjgcjq7//Oc/1913362vfe1rBqtKrL//+7/Xnj179Nxzz6mwsFBtbW3atGmT3G63qqqqTJeXUN/5znf085//XM8//7zy8vK0b98+rVq1Su+8845mzJhhuryB7CQkyT506JDpMsbcpUuXbEl2S0uL6VLG1M0332z/6Ec/Ml3GmOjp6bFvv/12+9VXX7V/7/d+z66qqjJdUkJ997vftRcsWGC6jDG1fft2e9myZabLGHdVVVX2V77yFfvq1aumS0mYtWvX2ps3b45pKy8vt7/xjW8Yqmhs/PrXv7YdDof9b//2bzHtCxYssB9++GFDVV1b0jymSUeBQECSNG3aNMOVjI1wOKwDBw7oypUrKioqMl3OmKisrNTatWu1atUq06WMmQsXLigvL08FBQX64z/+Y73//vumS0qoI0eOaPHixfra176mnJwcLVy4UE8++aTpssZUb2+v9u3bp82bN0e/B5YKli1bpv/4j//QL37xC0nSW2+9pddee02lpaWGK0usvr4+hcNhTZkyJaY9MzNTr732mqGqri1pHtOkG9u2VVNTo2XLlmnevHmmy0mos2fPqqioSP/7v/+rG2+8UYcOHdIdd9xhuqyEO3DggN58880J/Zx2tH73d39Xe/fu1e/8zu/o448/1ve//33dddddevvtt5WdnW26vIR4//33tXv3btXU1GjHjh366U9/qq1bt8rpdOqBBx4wXd6YOHz4sD777DN961vfMl1KQm3fvl2BQEBz5syRw+FQOBzWo48+qj/5kz8xXVpCZWVlqaioSI888ojmzp2r3Nxc7d+/X//1X/+l22+/3XR5gzN9a2YklAaPaf7iL/7CvvXWW+3Ozk7TpSRcKBSyL1y4YJ8+fdr+q7/6K/u3f/u37bffftt0WQnV0dFh5+Tk2GfOnIm2peJjmi+7fPmynZuba//TP/2T6VISZvLkyXZRUVFM25YtW+ylS5caqmjsrV692v7DP/xD02Uk3P79+22v12vv37/f/tnPfmbv3bvXnjZtmv3ss8+aLi3h3nvvPXv58uW2JNvhcNhLliyx77//fnvu3LmmSxsUYWQCevDBB22v12u///77pksZFyUlJfaf/dmfmS4joQ4dOhT9R6B/kWRblmU7HA67r6/PdIljZtWqVXZFRYXpMhImPz/f/va3vx3T9i//8i92Xl6eoYrG1i9/+Ut70qRJ9uHDh02XknBer9d+4oknYtoeeeQRe/bs2YYqGnuXL1+2P/roI9u2bfu+++6zS0tLDVc0OB7TTCC2bWvLli06dOiQmpubVVBQYLqkcWHbtkKhkOkyEqqkpERnz56Nadu0aZPmzJmj7du3p9wbJ/1CoZDOnTun4uJi06UkzFe/+tUBr9j/4he/0K233mqoorH1zDPPKCcnR2vXrjVdSsL9+te/1qRJsUMlHQ5HSr7a22/q1KmaOnWqPv30U73yyiv6h3/4B9MlDSppwsjly5f13nvvRdcvXryoM2fOaNq0acrPzzdYWeJUVlbqhRde0E9+8hNlZWWpu7tbkuR2u5WZmWm4usTYsWOH1qxZI5/Pp56eHh04cEDNzc06duyY6dISKisra8BYn6lTpyo7OzulxgBt27ZNZWVlys/P16VLl/T9739fwWBQGzduNF1awjz00EO666679Hd/93e677779NOf/lQNDQ1qaGgwXVrCXb16Vc8884w2btyojIykuTwMW1lZmR599FHl5+ersLBQ7e3t+ud//mdt3rzZdGkJ98orr8i2bc2ePVvvvfee/vIv/1KzZ8/Wpk2bTJc2OMN3ZobtxIkTtqQBy8aNG02XljCDnZ8k+5lnnjFdWsJs3rzZvvXWW+0bbrjBvuWWW+ySkhL7+PHjpssaF6k4ZuSP/uiPbI/HY0+ePNnOy8uzy8vLU278j23b9osvvmjPmzfPdjqd9pw5c+yGhgbTJY2JV155xZZknz9/3nQpYyIYDNpVVVV2fn6+PWXKFHvWrFn2ww8/bIdCIdOlJdyPf/xje9asWfYNN9xgT58+3a6srLQ/++wz02UNybJt2zYTgwAAAJJoOngAAJCaCCMAAMAowggAADCKMAIAAIwijAAAAKMIIwAAwCjCCAAAMIowAgAAjCKMAAAAowgjAADAKMIIAAAwijACAACM+j+y+6TljsHnWgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 导入绘制图表的库 matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "# scatter绘制的是散点图 scatter绘制的是散点图(x,y,color=\"?\")\n",
    "# X_train[行区间, 列区间]\n",
    "plt.scatter(X_train[y_train==0,0], X_train[y_train==0,1], color=\"red\")\n",
    "plt.scatter(X_train[y_train==1,0], X_train[y_train==1,1], color=\"blue\")\n",
    "plt.scatter(x[0], x[1],color=\"green\") \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54afd515-ed9d-473c-8be7-a2400a52f23d",
   "metadata": {},
   "source": [
    "**（3）计算样本之间的欧拉距离**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c9175916-d792-4c93-b477-94b738d4a6f1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4.807814343006513, 5.224098508750091, 6.746196350110584, 4.6967697972169224, 5.8305186039458645, 1.4935192351250712, 2.3513939670590434, 1.3753975288044795, 0.3040477783588936, 2.5727497158083397]\n"
     ]
    }
   ],
   "source": [
    "from math import sqrt\n",
    "distances = [sqrt(np.sum((x_train - x)**2)) for x_train in X_train]\n",
    "print(distances)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40c97beb-4f0e-46ea-b5df-437a5d952ae2",
   "metadata": {},
   "source": [
    "**（4）将距离从近到远排序，并返回排序后的原角标**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "adb7f3fe-258f-46fe-a41d-c41bb22c39d8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[8 7 5 6 9 3 0 1 4 2]\n"
     ]
    }
   ],
   "source": [
    "nearest = np.argsort(distances)\n",
    "print(nearest)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbd65594-0825-450e-ad03-96e2e5d72879",
   "metadata": {},
   "source": [
    "**（5）寻找前 6 个最近样本的分类**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ed703b3c-ebf5-482a-a0da-4503445c3248",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[np.int64(1), np.int64(1), np.int64(1), np.int64(1), np.int64(1), np.int64(0)]\n"
     ]
    }
   ],
   "source": [
    "k = 6\n",
    "topK_y = [y_train[i] for i in nearest[:k]]\n",
    "print(topK_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "131db495-e036-4752-b36a-af32161b11ab",
   "metadata": {},
   "source": [
    "**（6）投票分类结果**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "90018abb-9e00-4ed0-a9d4-2ef987ff198a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counter({np.int64(1): 5, np.int64(0): 1})\n",
      "[(np.int64(1), 5), (np.int64(0), 1)]\n",
      "[(np.int64(1), 5)]\n",
      "1\n"
     ]
    }
   ],
   "source": [
    "from collections import Counter\n",
    "votes = Counter(topK_y)\n",
    "print(votes) # 投票的结果\n",
    "print(votes.most_common()) # 投票的列表形式\n",
    "print(votes.most_common(1)) # 投票最多的结果 以列表的形式返回\n",
    "print(votes.most_common(1)[0][0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db7f4a4e-2a66-4177-8e2f-f35b04ad848e",
   "metadata": {},
   "source": [
    "## 1.3 机器学习流程"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f556d43-5b81-4e09-9a30-5e1abcd19409",
   "metadata": {},
   "source": [
    "机器学习流程可以概括为 **数据驱动的模型构建与应用**，核心流程分 4 步，如下图所示：\n",
    "\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250822125413999.png\" alt=\"image-20250822125413999\" style=\"zoom: 40%;\" />\n",
    "\n",
    "1. **准备数据：构建训练数据集**\n",
    "   - 整理特征数据 `X_train` ：用多维数据描述样本的属性，是模型 “学习” 的基础素材。\n",
    "   - 标记标签 `y_train` ：对应样本的结果，为模型提供 “学习目标”。\n",
    "   - 要求： `X_train` 与 `y_train` 样本数量一致，确保每个特征都有对应标签。\n",
    "2. **选择算法，训练模型（“fit” 过程 ）**\n",
    "   - 选机器学习算法：不同算法适配不同问题。\n",
    "   - 训练模型：算法”学习“ `X_train` 与 `y_train` 的关联规律，把规律 “固化” 成 模型。\n",
    "   - 部分算法的训练很轻量，甚至直接用原始数据做判断，但本质仍是从数据找规律。\n",
    "3. **输入新样本，预测（“predict” 过程 ）**\n",
    "   - 给模型输入 新样本 `x` ：特征数量 / 维度 需与 `X_train` 一致。\n",
    "   - 模型用 “学到的规律” 推理：分类问题输出类别，回归问题输出数值，完成 predict 。\n",
    "4. **迭代优化（可选但关键 ）**\n",
    "   - 用 测试数据 / 实际结果 验证模型：看预测准不准，若效果差，需调整（换算法、改参数、补数据等 ），让模型更 “聪明”。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7c1296e-52ed-420b-ad74-8702a935a562",
   "metadata": {},
   "source": [
    "## 1.4 K近邻算法封装"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e580306-0801-47b4-8085-1fc1354a9634",
   "metadata": {},
   "source": [
    "**自定义K近邻算法**\n",
    ">  [MyMLTools/neighbors.py](MyMLTools/neighbors.py) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a13e141-0320-4aec-83a9-1c5d48887c3f",
   "metadata": {},
   "source": [
    "**使用自定义的K近邻算法**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4fbdfb5d-9a9a-497b-9c76-ab4f00b480f1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1]\n"
     ]
    }
   ],
   "source": [
    "from MyMLTools.neighbors import KNeighborsClassifier\n",
    "knn = KNeighborsClassifier(k=6)\n",
    "knn.fit(X_train, y_train)\n",
    "X_predict = x.reshape(1,-1)\n",
    "print(knn.predict(X_predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c491c4a-497e-419a-bdf2-94be62544cbe",
   "metadata": {},
   "source": [
    "**使用sklearn中的K近邻算法**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "7db9dd4d-4f38-4dea-a3e5-402afc04bba9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "knn = KNeighborsClassifier(n_neighbors=6)\n",
    "knn.fit(X_train, y_train)\n",
    "y_predict = knn.predict(X_predict)\n",
    "print(y_predict)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "607e5df7-4f3e-4f58-8e11-4936e17075bc",
   "metadata": {},
   "source": [
    "## 1.5 训练集-测试集分割"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75cf5e6d-834e-4d08-8119-bf081d6f1f17",
   "metadata": {},
   "source": [
    "在机器学习中使用 train_test_split（训练集 - 测试集分割），核心目的是科学评估模型性能并避免过拟合\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250822171559187.png\" alt=\"image-20250822171559187\" style=\"zoom:30%;\" />\n",
    "\n",
    "具体可从以下 3 个关键维度理解：\n",
    "\n",
    "1. **模拟真实场景，评估泛化能力**\n",
    "    模型的最终目标是处理 未见过的数据。分割后，训练集用于让模型学习数据规律，测试集（完全不参与训练）则模拟 “真实未知数据”，通过测试集的预测结果（如准确率、误差），才能判断模型是否真的学会了通用规律，而非死记硬背。\n",
    "\n",
    "2. **避免过拟合，防止模型 “作弊”**\n",
    "    若用全量数据训练并评估，模型可能会 “记住” 数据中的噪声（如随机误差、个别异常值），表现为在训练数据上误差极小，但遇到新数据时误差骤增（即过拟合）。分割后，测试集的 “陌生性” 能有效暴露这种问题，帮助开发者调整模型（如简化结构、增加数据）。\n",
    "\n",
    "3. **指导模型优化，避免盲目调参**\n",
    "    没有独立测试集时，开发者可能会根据 “全量数据的表现” 反复调整参数（如学习率、树的深度），导致模型 “适配” 了当前所有数据，失去对新数据的适应性。测试集提供了一个 “客观标尺”，让优化过程始终围绕 “提升泛化能力” 展开，而非单纯追求训练数据上的完美表现。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5f120d6-827a-41ed-8b53-326fec38511b",
   "metadata": {},
   "source": [
    "**（1）准备鸢尾花数据**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b8bc38e9-c67d-4497-9116-1d0605560168",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[5.1 3.5 1.4 0.2]\n",
      " [4.9 3.  1.4 0.2]\n",
      " [4.7 3.2 1.3 0.2]\n",
      " [4.6 3.1 1.5 0.2]\n",
      " [5.  3.6 1.4 0.2]\n",
      " [5.4 3.9 1.7 0.4]\n",
      " [4.6 3.4 1.4 0.3]\n",
      " [5.  3.4 1.5 0.2]\n",
      " [4.4 2.9 1.4 0.2]\n",
      " [4.9 3.1 1.5 0.1]\n",
      " [5.4 3.7 1.5 0.2]\n",
      " [4.8 3.4 1.6 0.2]\n",
      " [4.8 3.  1.4 0.1]\n",
      " [4.3 3.  1.1 0.1]\n",
      " [5.8 4.  1.2 0.2]\n",
      " [5.7 4.4 1.5 0.4]\n",
      " [5.4 3.9 1.3 0.4]\n",
      " [5.1 3.5 1.4 0.3]\n",
      " [5.7 3.8 1.7 0.3]\n",
      " [5.1 3.8 1.5 0.3]\n",
      " [5.4 3.4 1.7 0.2]\n",
      " [5.1 3.7 1.5 0.4]\n",
      " [4.6 3.6 1.  0.2]\n",
      " [5.1 3.3 1.7 0.5]\n",
      " [4.8 3.4 1.9 0.2]\n",
      " [5.  3.  1.6 0.2]\n",
      " [5.  3.4 1.6 0.4]\n",
      " [5.2 3.5 1.5 0.2]\n",
      " [5.2 3.4 1.4 0.2]\n",
      " [4.7 3.2 1.6 0.2]\n",
      " [4.8 3.1 1.6 0.2]\n",
      " [5.4 3.4 1.5 0.4]\n",
      " [5.2 4.1 1.5 0.1]\n",
      " [5.5 4.2 1.4 0.2]\n",
      " [4.9 3.1 1.5 0.2]\n",
      " [5.  3.2 1.2 0.2]\n",
      " [5.5 3.5 1.3 0.2]\n",
      " [4.9 3.6 1.4 0.1]\n",
      " [4.4 3.  1.3 0.2]\n",
      " [5.1 3.4 1.5 0.2]\n",
      " [5.  3.5 1.3 0.3]\n",
      " [4.5 2.3 1.3 0.3]\n",
      " [4.4 3.2 1.3 0.2]\n",
      " [5.  3.5 1.6 0.6]\n",
      " [5.1 3.8 1.9 0.4]\n",
      " [4.8 3.  1.4 0.3]\n",
      " [5.1 3.8 1.6 0.2]\n",
      " [4.6 3.2 1.4 0.2]\n",
      " [5.3 3.7 1.5 0.2]\n",
      " [5.  3.3 1.4 0.2]\n",
      " [7.  3.2 4.7 1.4]\n",
      " [6.4 3.2 4.5 1.5]\n",
      " [6.9 3.1 4.9 1.5]\n",
      " [5.5 2.3 4.  1.3]\n",
      " [6.5 2.8 4.6 1.5]\n",
      " [5.7 2.8 4.5 1.3]\n",
      " [6.3 3.3 4.7 1.6]\n",
      " [4.9 2.4 3.3 1. ]\n",
      " [6.6 2.9 4.6 1.3]\n",
      " [5.2 2.7 3.9 1.4]\n",
      " [5.  2.  3.5 1. ]\n",
      " [5.9 3.  4.2 1.5]\n",
      " [6.  2.2 4.  1. ]\n",
      " [6.1 2.9 4.7 1.4]\n",
      " [5.6 2.9 3.6 1.3]\n",
      " [6.7 3.1 4.4 1.4]\n",
      " [5.6 3.  4.5 1.5]\n",
      " [5.8 2.7 4.1 1. ]\n",
      " [6.2 2.2 4.5 1.5]\n",
      " [5.6 2.5 3.9 1.1]\n",
      " [5.9 3.2 4.8 1.8]\n",
      " [6.1 2.8 4.  1.3]\n",
      " [6.3 2.5 4.9 1.5]\n",
      " [6.1 2.8 4.7 1.2]\n",
      " [6.4 2.9 4.3 1.3]\n",
      " [6.6 3.  4.4 1.4]\n",
      " [6.8 2.8 4.8 1.4]\n",
      " [6.7 3.  5.  1.7]\n",
      " [6.  2.9 4.5 1.5]\n",
      " [5.7 2.6 3.5 1. ]\n",
      " [5.5 2.4 3.8 1.1]\n",
      " [5.5 2.4 3.7 1. ]\n",
      " [5.8 2.7 3.9 1.2]\n",
      " [6.  2.7 5.1 1.6]\n",
      " [5.4 3.  4.5 1.5]\n",
      " [6.  3.4 4.5 1.6]\n",
      " [6.7 3.1 4.7 1.5]\n",
      " [6.3 2.3 4.4 1.3]\n",
      " [5.6 3.  4.1 1.3]\n",
      " [5.5 2.5 4.  1.3]\n",
      " [5.5 2.6 4.4 1.2]\n",
      " [6.1 3.  4.6 1.4]\n",
      " [5.8 2.6 4.  1.2]\n",
      " [5.  2.3 3.3 1. ]\n",
      " [5.6 2.7 4.2 1.3]\n",
      " [5.7 3.  4.2 1.2]\n",
      " [5.7 2.9 4.2 1.3]\n",
      " [6.2 2.9 4.3 1.3]\n",
      " [5.1 2.5 3.  1.1]\n",
      " [5.7 2.8 4.1 1.3]\n",
      " [6.3 3.3 6.  2.5]\n",
      " [5.8 2.7 5.1 1.9]\n",
      " [7.1 3.  5.9 2.1]\n",
      " [6.3 2.9 5.6 1.8]\n",
      " [6.5 3.  5.8 2.2]\n",
      " [7.6 3.  6.6 2.1]\n",
      " [4.9 2.5 4.5 1.7]\n",
      " [7.3 2.9 6.3 1.8]\n",
      " [6.7 2.5 5.8 1.8]\n",
      " [7.2 3.6 6.1 2.5]\n",
      " [6.5 3.2 5.1 2. ]\n",
      " [6.4 2.7 5.3 1.9]\n",
      " [6.8 3.  5.5 2.1]\n",
      " [5.7 2.5 5.  2. ]\n",
      " [5.8 2.8 5.1 2.4]\n",
      " [6.4 3.2 5.3 2.3]\n",
      " [6.5 3.  5.5 1.8]\n",
      " [7.7 3.8 6.7 2.2]\n",
      " [7.7 2.6 6.9 2.3]\n",
      " [6.  2.2 5.  1.5]\n",
      " [6.9 3.2 5.7 2.3]\n",
      " [5.6 2.8 4.9 2. ]\n",
      " [7.7 2.8 6.7 2. ]\n",
      " [6.3 2.7 4.9 1.8]\n",
      " [6.7 3.3 5.7 2.1]\n",
      " [7.2 3.2 6.  1.8]\n",
      " [6.2 2.8 4.8 1.8]\n",
      " [6.1 3.  4.9 1.8]\n",
      " [6.4 2.8 5.6 2.1]\n",
      " [7.2 3.  5.8 1.6]\n",
      " [7.4 2.8 6.1 1.9]\n",
      " [7.9 3.8 6.4 2. ]\n",
      " [6.4 2.8 5.6 2.2]\n",
      " [6.3 2.8 5.1 1.5]\n",
      " [6.1 2.6 5.6 1.4]\n",
      " [7.7 3.  6.1 2.3]\n",
      " [6.3 3.4 5.6 2.4]\n",
      " [6.4 3.1 5.5 1.8]\n",
      " [6.  3.  4.8 1.8]\n",
      " [6.9 3.1 5.4 2.1]\n",
      " [6.7 3.1 5.6 2.4]\n",
      " [6.9 3.1 5.1 2.3]\n",
      " [5.8 2.7 5.1 1.9]\n",
      " [6.8 3.2 5.9 2.3]\n",
      " [6.7 3.3 5.7 2.5]\n",
      " [6.7 3.  5.2 2.3]\n",
      " [6.3 2.5 5.  1.9]\n",
      " [6.5 3.  5.2 2. ]\n",
      " [6.2 3.4 5.4 2.3]\n",
      " [5.9 3.  5.1 1.8]]\n",
      "[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2]\n",
      "(150, 4)\n",
      "(150,)\n"
     ]
    }
   ],
   "source": [
    "# 导入鸢尾花数据集\n",
    "from sklearn import datasets\n",
    "iris = datasets.load_iris() # 导入鸢尾花数据集\n",
    "X = iris.data # 特征矩阵\n",
    "y = iris.target # 标签向量\n",
    "print(X)\n",
    "print(y)\n",
    "print(X.shape)\n",
    "print(y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53ce4f9c-2ee9-4519-ab0f-714d2ef33773",
   "metadata": {},
   "source": [
    "**（2）切割数据**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "38f58570-2b76-4e67-be4f-84cc2282ecb7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 21 121  17 133  48  18  81   5  93  26  27  87  11 119  86  56 131 148\n",
      "  16  83  91  43  34  36  63   0 100  60 129 124  66   8 146  58  96 106\n",
      "  61 120  35 130   3  73  57  99  25  10 105  74  85  41  28 113  84   1\n",
      " 134  89 107 116  37 140  38  15  53  22  52 110  75  79 138 118 102  71\n",
      "  14  32  44 128  30 127  50  20 125  24  19  64  40 147 122  47   2  72\n",
      "  23 111  59 149  97 104 109 137  51 132  12 108 139  13  33   9  82 141\n",
      "  69 103 145 142  77  88   4  98 126  90  70 101  39  31  76 144  92 135\n",
      "  55   7 117 114  45 123 143  46  49  80  54  29  65  67  42 115  62   6\n",
      "  68 136  95  78  94 112]\n",
      "30\n"
     ]
    }
   ],
   "source": [
    "shuffle_indexes = np.random.permutation(len(X))\n",
    "print(shuffle_indexes)\n",
    "test_radio = 0.2\n",
    "test_size = int(len(X) * test_radio)\n",
    "print(test_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "5d5acdad-c1e6-4e3a-b422-91ade7ce2d1f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(120, 4)\n",
      "(120,)\n",
      "(30, 4)\n",
      "(30,)\n"
     ]
    }
   ],
   "source": [
    "test_indexes = shuffle_indexes[:test_size]\n",
    "train_indexes = shuffle_indexes[test_size:]\n",
    "X_train = X[train_indexes]\n",
    "y_train = y[train_indexes]\n",
    "X_test = X[test_indexes]\n",
    "y_test = y[test_indexes]\n",
    "print(X_train.shape)\n",
    "print(y_train.shape)\n",
    "print(X_test.shape)\n",
    "print(y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "026ffe94-9527-49e1-8f75-402d60800046",
   "metadata": {},
   "source": [
    "**（3）自定义train_test_split**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f710cfae-8642-4dd6-9b33-07fed65f6565",
   "metadata": {},
   "source": [
    "> [MyMLTools/train_test_split.py](MyMLTools/train_test_split.py)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d901f51f-a386-4701-b84b-a6a764aaa7a8",
   "metadata": {},
   "source": [
    "**（4）使用自定义train_test_split**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0deffa6d-c4ea-4102-ab52-94b40867b90f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(120, 4)\n",
      "(120,)\n",
      "(30, 4)\n",
      "(30,)\n"
     ]
    }
   ],
   "source": [
    "from MyMLTools.train_test_split import train_test_split\n",
    "X_train, X_test,y_train, y_test = train_test_split(X,y,test_radio=0.2,seed=123)\n",
    "print(X_train.shape)\n",
    "print(y_train.shape)\n",
    "print(X_test.shape)\n",
    "print(y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c12a1378-235b-4bca-983b-54bc9dd9d611",
   "metadata": {},
   "source": [
    "**（5）利用训练-测试数据集训练模型**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "61c2a0db-31b9-423a-a701-b2d9b3d3096e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2 2 2 1 0 1 1 0 0 1 2 0 1 2 2 2 0 0 1 0 0 2 0 2 0 0 0 2 2 0]\n",
      "[1 2 2 1 0 2 1 0 0 1 2 0 1 2 2 2 0 0 1 0 0 2 0 2 0 0 0 2 2 0]\n",
      "0.9333333333333333\n"
     ]
    }
   ],
   "source": [
    "from MyMLTools.neighbors import KNeighborsClassifier\n",
    "knn = KNeighborsClassifier(k=6)\n",
    "knn.fit(X_train,y_train)\n",
    "y_predict = knn.predict(X_test)\n",
    "print(y_predict)\n",
    "print(y_test)\n",
    "print(sum(y_predict == y_test) / len(y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c952e63f-5999-416c-9e89-f641b53d9458",
   "metadata": {},
   "source": [
    "**（6）使用sklearn中的train_test_split**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "630af53f-ac3c-4df0-9894-0e28ae2ea0f0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,random_state=456)\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "knn = KNeighborsClassifier(n_neighbors=6)\n",
    "knn.fit(X_train,y_train)\n",
    "y_predict = knn.predict(X_test)\n",
    "print(sum(y_predict == y_test) / len(y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d220067-3ce9-4c74-b871-2635b303dff9",
   "metadata": {},
   "source": [
    "## 1.6 分类准确度"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6a4273e-70ae-4198-8410-364f64be744c",
   "metadata": {},
   "source": [
    "在机器学习分类任务中，准确度（Accuracy） 是最基础、最直观的模型性能评估指标，核心用于衡量模型 “预测结果与真实情况一致的比例”，本质是对模型整体预测正确性的量化描述。\n",
    "\n",
    "`准确度 = 模型预测正确的样本数量 ÷ 总样本数量`\n",
    "\n",
    "它直接回答了 “在所有测试样本中，模型猜对了多少” 这一问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "605b4543-f3cc-4bdb-a7d7-f617d2671c4c",
   "metadata": {},
   "source": [
    "**（1）准备手写数字数据**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "1102a6de-183c-4073-b676-a90f5ee22b31",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _digits_dataset:\n",
      "\n",
      "Optical recognition of handwritten digits dataset\n",
      "--------------------------------------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      ":Number of Instances: 1797\n",
      ":Number of Attributes: 64\n",
      ":Attribute Information: 8x8 image of integer pixels in the range 0..16.\n",
      ":Missing Attribute Values: None\n",
      ":Creator: E. Alpaydin (alpaydin '@' boun.edu.tr)\n",
      ":Date: July; 1998\n",
      "\n",
      "This is a copy of the test set of the UCI ML hand-written digits datasets\n",
      "https://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits\n",
      "\n",
      "The data set contains images of hand-written digits: 10 classes where\n",
      "each class refers to a digit.\n",
      "\n",
      "Preprocessing programs made available by NIST were used to extract\n",
      "normalized bitmaps of handwritten digits from a preprinted form. From a\n",
      "total of 43 people, 30 contributed to the training set and different 13\n",
      "to the test set. 32x32 bitmaps are divided into nonoverlapping blocks of\n",
      "4x4 and the number of on pixels are counted in each block. This generates\n",
      "an input matrix of 8x8 where each element is an integer in the range\n",
      "0..16. This reduces dimensionality and gives invariance to small\n",
      "distortions.\n",
      "\n",
      "For info on NIST preprocessing routines, see M. D. Garris, J. L. Blue, G.\n",
      "T. Candela, D. L. Dimmick, J. Geist, P. J. Grother, S. A. Janet, and C.\n",
      "L. Wilson, NIST Form-Based Handprint Recognition System, NISTIR 5469,\n",
      "1994.\n",
      "\n",
      ".. dropdown:: References\n",
      "\n",
      "  - C. Kaynak (1995) Methods of Combining Multiple Classifiers and Their\n",
      "    Applications to Handwritten Digit Recognition, MSc Thesis, Institute of\n",
      "    Graduate Studies in Science and Engineering, Bogazici University.\n",
      "  - E. Alpaydin, C. Kaynak (1998) Cascading Classifiers, Kybernetika.\n",
      "  - Ken Tang and Ponnuthurai N. Suganthan and Xi Yao and A. Kai Qin.\n",
      "    Linear dimensionalityreduction using relevance weighted LDA. School of\n",
      "    Electrical and Electronic Engineering Nanyang Technological University.\n",
      "    2005.\n",
      "  - Claudio Gentile. A New Approximate Maximal Margin Classification\n",
      "    Algorithm. NIPS. 2000.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 导入手写数字的数据\n",
    "from sklearn import datasets\n",
    "digits = datasets.load_digits()\n",
    "print(digits.DESCR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "97db3c8b-ad37-4db1-a6e3-993cc50c5f41",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = digits.data\n",
    "y = digits.target"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3f17427-8942-4590-b871-7cd99367608f",
   "metadata": {},
   "source": [
    "**（2）查看一个手写数字数据**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "1dce9c9e-db0f-4299-915b-9d518bb0eb8d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.  0.  5. ...  0.  0.  0.]\n",
      " [ 0.  0.  0. ... 10.  0.  0.]\n",
      " [ 0.  0.  0. ... 16.  9.  0.]\n",
      " ...\n",
      " [ 0.  0.  1. ...  6.  0.  0.]\n",
      " [ 0.  0.  2. ... 12.  0.  0.]\n",
      " [ 0.  0. 10. ... 12.  1.  0.]]\n",
      "[0 1 2 ... 8 9 8]\n",
      "[ 0.  0.  0.  2. 16. 15.  3.  0.  0.  0.  0.  8. 16. 16.  4.  0.  0.  0.\n",
      "  9. 16. 16. 14.  0.  0.  0.  7. 16. 16. 16. 12.  0.  0.  0.  0.  0.  8.\n",
      " 16. 12.  0.  0.  0.  0.  0.  7. 16. 12.  0.  0.  0.  0.  0.  4. 16. 16.\n",
      "  7.  0.  0.  0.  0.  0. 13. 16.  7.  0.]\n",
      "1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAGdCAYAAAAv9mXmAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAF8JJREFUeJzt3X9s1IX9x/HXwcmh0B6CFNtwlAaJ/CgFbJkr4ETBJg0SzTamC7I65rLOgtTGxFX/kP3i2B9bwKjNykiVECxZJsjiAEsmxcV0K9VOhgZhEHoKrIHIHTTZEdvP949vvNghpZ9r3/3wOZ6P5JN4l89xr5jK089d2ws4juMIAIBBNszrAQCAzERgAAAmCAwAwASBAQCYIDAAABMEBgBggsAAAEwQGACAieBQP2FPT49Onz6trKwsBQKBoX56AMAAOI6jixcvKi8vT8OG9X2NMuSBOX36tCKRyFA/LQBgEMViMU2cOLHPc4Y8MFlZWZL+f1x2dvZQPz185tSpU15PSEtRUZHXE9Li1/8m33rrLa8npM1vXyuJREKRSCT1d3lfhjwwX74slp2d7dsvZgyd/nwRY/D49WXr0aNHez0hbX79e7A/Xyu8yQ8AMEFgAAAmCAwAwASBAQCYIDAAABMEBgBggsAAAEwQGACACQIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgIm0AvPKK6+ooKBAI0eOVHFxsd59993B3gUA8DnXgdmxY4eqq6v1/PPP64MPPtA999yj8vJydXR0WOwDAPiU68D87ne/049+9CM98cQTmj59ujZu3KhIJKK6ujqLfQAAn3IVmMuXL6utrU1lZWW97i8rK9N77733tY9JJpNKJBK9DgBA5nMVmHPnzqm7u1sTJkzodf+ECRN09uzZr31MNBpVOBxOHZFIJP21AADfSOtN/kAg0Ou24zhX3Pel2tpaxePx1BGLxdJ5SgCAzwTdnHzbbbdp+PDhV1ytdHZ2XnFV86VQKKRQKJT+QgCAL7m6ghkxYoSKi4vV1NTU6/6mpibNnz9/UIcBAPzN1RWMJNXU1GjlypUqKSlRaWmp6uvr1dHRocrKSot9AACfch2YRx55ROfPn9cvfvELnTlzRoWFhfrLX/6i/Px8i30AAJ9yHRhJevLJJ/Xkk08O9hYAQAbhd5EBAEwQGACACQIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADABIEBAJggMAAAE2l9HgwwVF599VWvJ9xQ4vG41xPScuHCBa8n4GtwBQMAMEFgAAAmCAwAwASBAQCYIDAAABMEBgBggsAAAEwQGACACQIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADAhOvAHDx4UMuWLVNeXp4CgYB27dplMAsA4HeuA9PV1aXZs2frpZdestgDAMgQQbcPKC8vV3l5ucUWAEAGcR0Yt5LJpJLJZOp2IpGwfkoAwHXA/E3+aDSqcDicOiKRiPVTAgCuA+aBqa2tVTweTx2xWMz6KQEA1wHzl8hCoZBCoZD10wAArjP8HAwAwITrK5hLly7p+PHjqdsnT55Ue3u7xo4dq0mTJg3qOACAf7kOzKFDh3TfffelbtfU1EiSKioq9Oqrrw7aMACAv7kOzKJFi+Q4jsUWAEAG4T0YAIAJAgMAMEFgAAAmCAwAwASBAQCYIDAAABMEBgBggsAAAEwQGACACQIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYML158HAf6qrq72ekLZNmzZ5PeGGkp+f7/WEtCxatMjrCfgaXMEAAEwQGACACQIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADABIEBAJggMAAAEwQGAGCCwAAATBAYAIAJAgMAMOEqMNFoVPPmzVNWVpZycnL08MMP6+jRo1bbAAA+5iowzc3NqqqqUktLi5qamvTFF1+orKxMXV1dVvsAAD4VdHPy3r17e91uaGhQTk6O2tra9K1vfWtQhwEA/M1VYP5XPB6XJI0dO/aq5ySTSSWTydTtRCIxkKcEAPhE2m/yO46jmpoaLVy4UIWFhVc9LxqNKhwOp45IJJLuUwIAfCTtwKxevVoffvihXn/99T7Pq62tVTweTx2xWCzdpwQA+EhaL5GtWbNGu3fv1sGDBzVx4sQ+zw2FQgqFQmmNAwD4l6vAOI6jNWvWaOfOnTpw4IAKCgqsdgEAfM5VYKqqqrR9+3a9+eabysrK0tmzZyVJ4XBYN998s8lAAIA/uXoPpq6uTvF4XIsWLVJubm7q2LFjh9U+AIBPuX6JDACA/uB3kQEATBAYAIAJAgMAMEFgAAAmCAwAwASBAQCYIDAAABMEBgBggsAAAEwQGACACQIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYMLVB47d6DZu3Oj1hLRs2rTJ6wnwicmTJ3s9ARmEKxgAgAkCAwAwQWAAACYIDADABIEBAJggMAAAEwQGAGCCwAAATBAYAIAJAgMAMEFgAAAmCAwAwASBAQCYIDAAABMEBgBggsAAAEwQGACACQIDADDhKjB1dXUqKipSdna2srOzVVpaqj179lhtAwD4mKvATJw4URs2bNChQ4d06NAh3X///XrooYd05MgRq30AAJ8Kujl52bJlvW7/+te/Vl1dnVpaWjRz5sxBHQYA8DdXgfmq7u5u/fGPf1RXV5dKS0uvel4ymVQymUzdTiQS6T4lAMBHXL/Jf/jwYY0ePVqhUEiVlZXauXOnZsyYcdXzo9GowuFw6ohEIgMaDADwB9eBufPOO9Xe3q6Wlhb99Kc/VUVFhT766KOrnl9bW6t4PJ46YrHYgAYDAPzB9UtkI0aM0B133CFJKikpUWtrqzZt2qTf//73X3t+KBRSKBQa2EoAgO8M+OdgHMfp9R4LAACSyyuY5557TuXl5YpEIrp48aIaGxt14MAB7d2712ofAMCnXAXmP//5j1auXKkzZ84oHA6rqKhIe/fu1QMPPGC1DwDgU64Cs2XLFqsdAIAMw+8iAwCYIDAAABMEBgBggsAAAEwQGACACQIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADAhKsPHLvRVVdXez0hLX7dLUnr1q3zekJafv7zn3s9AfAcVzAAABMEBgBggsAAAEwQGACACQIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADABIEBAJggMAAAEwQGAGBiQIGJRqMKBAK+/kheAICNtAPT2tqq+vp6FRUVDeYeAECGSCswly5d0ooVK7R582bdeuutg70JAJAB0gpMVVWVli5dqiVLlgz2HgBAhgi6fUBjY6Pef/99tba29uv8ZDKpZDKZup1IJNw+JQDAh1xdwcRiMa1du1bbtm3TyJEj+/WYaDSqcDicOiKRSFpDAQD+4iowbW1t6uzsVHFxsYLBoILBoJqbm/Xiiy8qGAyqu7v7isfU1tYqHo+njlgsNmjjAQDXL1cvkS1evFiHDx/udd8Pf/hDTZs2Tc8++6yGDx9+xWNCoZBCodDAVgIAfMdVYLKyslRYWNjrvlGjRmncuHFX3A8AuLHxk/wAABOuv4vsfx04cGAQZgAAMg1XMAAAEwQGAGCCwAAATBAYAIAJAgMAMEFgAAAmCAwAwASBAQCYIDAAABMEBgBggsAAAEwQGACACQIDADBBYAAAJggMAMAEgQEAmBjwB44BlsaMGeP1BABp4goGAGCCwAAATBAYAIAJAgMAMEFgAAAmCAwAwASBAQCYIDAAABMEBgBggsAAAEwQGACACQIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAlXgVm3bp0CgUCv4/bbb7faBgDwsaDbB8ycOVP79+9P3R4+fPigDgIAZAbXgQkGg1y1AACuyfV7MMeOHVNeXp4KCgr06KOP6sSJE32en0wmlUgkeh0AgMznKjB33323tm7dqn379mnz5s06e/as5s+fr/Pnz1/1MdFoVOFwOHVEIpEBjwYAXP9cBaa8vFzf+c53NGvWLC1ZskRvvfWWJOm111676mNqa2sVj8dTRywWG9hiAIAvuH4P5qtGjRqlWbNm6dixY1c9JxQKKRQKDeRpAAA+NKCfg0kmk/r444+Vm5s7WHsAABnCVWCeeeYZNTc36+TJk/r73/+u7373u0okEqqoqLDaBwDwKVcvkX366af6/ve/r3Pnzmn8+PH65je/qZaWFuXn51vtAwD4lKvANDY2Wu0AAGQYfhcZAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADABIEBAJggMAAAEwQGAGCCwAAATBAYAIAJAgMAMOHq82CAoTZnzhyvJ9xQmpubvZ6Qlo0bN3o9IW3V1dVeTzDDFQwAwASBAQCYIDAAABMEBgBggsAAAEwQGACACQIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADABIEBAJggMAAAE64D89lnn+mxxx7TuHHjdMstt2jOnDlqa2uz2AYA8LGgm5M///xzLViwQPfdd5/27NmjnJwc/fvf/9aYMWOM5gEA/MpVYH7zm98oEomooaEhdd/kyZMHexMAIAO4eols9+7dKikp0fLly5WTk6O5c+dq8+bNfT4mmUwqkUj0OgAAmc9VYE6cOKG6ujpNnTpV+/btU2VlpZ566ilt3br1qo+JRqMKh8OpIxKJDHg0AOD65yowPT09uuuuu7R+/XrNnTtXP/nJT/TjH/9YdXV1V31MbW2t4vF46ojFYgMeDQC4/rkKTG5urmbMmNHrvunTp6ujo+OqjwmFQsrOzu51AAAyn6vALFiwQEePHu113yeffKL8/PxBHQUA8D9XgXn66afV0tKi9evX6/jx49q+fbvq6+tVVVVltQ8A4FOuAjNv3jzt3LlTr7/+ugoLC/XLX/5SGzdu1IoVK6z2AQB8ytXPwUjSgw8+qAcffNBiCwAgg/C7yAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADABIEBAJggMAAAEwQGAGCCwAAATBAYAIAJAgMAMOH6A8eAobRo0SKvJ6Rl9uzZXk9Iyz//+U+vJyCDcAUDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADABIEBAJggMAAAEwQGAGCCwAAATBAYAIAJAgMAMEFgAAAmXAVm8uTJCgQCVxxVVVVW+wAAPhV0c3Jra6u6u7tTt//1r3/pgQce0PLlywd9GADA31wFZvz48b1ub9iwQVOmTNG99947qKMAAP7nKjBfdfnyZW3btk01NTUKBAJXPS+ZTCqZTKZuJxKJdJ8SAOAjab/Jv2vXLl24cEGPP/54n+dFo1GFw+HUEYlE0n1KAICPpB2YLVu2qLy8XHl5eX2eV1tbq3g8njpisVi6TwkA8JG0XiI7deqU9u/frzfeeOOa54ZCIYVCoXSeBgDgY2ldwTQ0NCgnJ0dLly4d7D0AgAzhOjA9PT1qaGhQRUWFgsG0v0cAAJDhXAdm//796ujo0KpVqyz2AAAyhOtLkLKyMjmOY7EFAJBB+F1kAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADABIEBAJggMAAAEwQGAGCCwAAATBAYAIAJAgMAMEFgAAAmCAwAwMSQfyTll58lk0gkhvqpgSHT3d3t9YQbyn//+1+vJ6TNb38Xfrm3P58LFnCG+NPDPv30U0UikaF8SgDAIIvFYpo4cWKf5wx5YHp6enT69GllZWUpEAgM6p+dSCQUiUQUi8WUnZ09qH+2JXYPLXYPPb9uZ/eVHMfRxYsXlZeXp2HD+n6XZchfIhs2bNg1qzdQ2dnZvvpi+BK7hxa7h55ft7O7t3A43K/zeJMfAGCCwAAATGRUYEKhkF544QWFQiGvp7jC7qHF7qHn1+3sHpghf5MfAHBjyKgrGADA9YPAAABMEBgAgAkCAwAwkTGBeeWVV1RQUKCRI0equLhY7777rteTrungwYNatmyZ8vLyFAgEtGvXLq8n9Us0GtW8efOUlZWlnJwcPfzwwzp69KjXs66prq5ORUVFqR8+Ky0t1Z49e7ye5Vo0GlUgEFB1dbXXU/q0bt06BQKBXsftt9/u9ax++eyzz/TYY49p3LhxuuWWWzRnzhy1tbV5PeuaJk+efMW/80AgoKqqKk/2ZERgduzYoerqaj3//PP64IMPdM8996i8vFwdHR1eT+tTV1eXZs+erZdeesnrKa40NzerqqpKLS0tampq0hdffKGysjJ1dXV5Pa1PEydO1IYNG3To0CEdOnRI999/vx566CEdOXLE62n91traqvr6ehUVFXk9pV9mzpypM2fOpI7Dhw97PemaPv/8cy1YsEA33XST9uzZo48++ki//e1vNWbMGK+nXVNra2uvf99NTU2SpOXLl3szyMkA3/jGN5zKyspe902bNs352c9+5tEi9yQ5O3fu9HpGWjo7Ox1JTnNzs9dTXLv11ludP/zhD17P6JeLFy86U6dOdZqampx7773XWbt2rdeT+vTCCy84s2fP9nqGa88++6yzcOFCr2cMirVr1zpTpkxxenp6PHl+31/BXL58WW1tbSorK+t1f1lZmd577z2PVt1Y4vG4JGns2LEeL+m/7u5uNTY2qqurS6WlpV7P6ZeqqiotXbpUS5Ys8XpKvx07dkx5eXkqKCjQo48+qhMnTng96Zp2796tkpISLV++XDk5OZo7d642b97s9SzXLl++rG3btmnVqlWD/ouF+8v3gTl37py6u7s1YcKEXvdPmDBBZ8+e9WjVjcNxHNXU1GjhwoUqLCz0es41HT58WKNHj1YoFFJlZaV27typGTNmeD3rmhobG/X+++8rGo16PaXf7r77bm3dulX79u3T5s2bdfbsWc2fP1/nz5/3elqfTpw4obq6Ok2dOlX79u1TZWWlnnrqKW3dutXraa7s2rVLFy5c0OOPP+7ZhiH/bcpW/rfQjuN4Vu0byerVq/Xhhx/qb3/7m9dT+uXOO+9Ue3u7Lly4oD/96U+qqKhQc3PzdR2ZWCymtWvX6u2339bIkSO9ntNv5eXlqX+eNWuWSktLNWXKFL322muqqanxcFnfenp6VFJSovXr10uS5s6dqyNHjqiurk4/+MEPPF7Xf1u2bFF5ebny8vI82+D7K5jbbrtNw4cPv+JqpbOz84qrGgyuNWvWaPfu3XrnnXfMP4JhsIwYMUJ33HGHSkpKFI1GNXv2bG3atMnrWX1qa2tTZ2eniouLFQwGFQwG1dzcrBdffFHBYNA3n545atQozZo1S8eOHfN6Sp9yc3Ov+B+O6dOnX/ffNPRVp06d0v79+/XEE094usP3gRkxYoSKi4tT3y3xpaamJs2fP9+jVZnNcRytXr1ab7zxhv7617+qoKDA60lpcxxHyWTS6xl9Wrx4sQ4fPqz29vbUUVJSohUrVqi9vV3Dhw/3emK/JJNJffzxx8rNzfV6Sp8WLFhwxbfdf/LJJ8rPz/dokXsNDQ3KycnR0qVLPd2RES+R1dTUaOXKlSopKVFpaanq6+vV0dGhyspKr6f16dKlSzp+/Hjq9smTJ9Xe3q6xY8dq0qRJHi7rW1VVlbZv364333xTWVlZqavHcDism2++2eN1V/fcc8+pvLxckUhEFy9eVGNjow4cOKC9e/d6Pa1PWVlZV7y/NWrUKI0bN+66ft/rmWee0bJlyzRp0iR1dnbqV7/6lRKJhCoqKrye1qenn35a8+fP1/r16/W9731P//jHP1RfX6/6+nqvp/VLT0+PGhoaVFFRoWDQ47/iPfneNQMvv/yyk5+f74wYMcK56667fPEts++8844j6YqjoqLC62l9+rrNkpyGhgavp/Vp1apVqa+R8ePHO4sXL3befvttr2elxQ/fpvzII484ubm5zk033eTk5eU53/72t50jR454Patf/vznPzuFhYVOKBRypk2b5tTX13s9qd/27dvnSHKOHj3q9RSHX9cPADDh+/dgAADXJwIDADBBYAAAJggMAMAEgQEAmCAwAAATBAYAYILAAABMEBgAgAkCAwAwQWAAACYIDADAxP8BOId+dWK8hjQAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(X)\n",
    "print(y)\n",
    "print(X[456])\n",
    "print(y[456])\n",
    "some_digit = X[456]\n",
    "some_digit_image = some_digit.reshape(8,8)\n",
    "import matplotlib\n",
    "plt.imshow(some_digit_image, cmap=matplotlib.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b612e3d0-fd19-4bec-aaf4-fddbba7c10ee",
   "metadata": {},
   "source": [
    "**（3）切割训练集和测试集并训练模型**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "727eff2c-b83a-4de8-afe2-5abc5d45cfeb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9860724233983287\n"
     ]
    }
   ],
   "source": [
    "from MyMLTools.train_test_split import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_radio=0.2, seed=666)\n",
    "from MyMLTools.neighbors import KNeighborsClassifier\n",
    "knn = KNeighborsClassifier(k=6)\n",
    "knn.fit(X_train, y_train)\n",
    "y_predict = knn.predict(X_test)\n",
    "print(sum(y_predict == y_test) / len(y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f519178a-7daf-44be-a185-6b7e5c806a69",
   "metadata": {},
   "source": [
    "**（4）自定义分类准确度**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35f8da3e-43b4-4c7b-b79d-3b427a1afc1e",
   "metadata": {},
   "source": [
    "> [MyMLTools/metrics.py](MyMLTools/metrics.py)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dc7222b-7a30-40fe-afa9-48b5e8f9a56f",
   "metadata": {},
   "source": [
    "**（5）使用自定义分类准确度**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "9856a85e-66fb-4794-a5ab-7ab8f018b780",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9860724233983287\n"
     ]
    }
   ],
   "source": [
    "from MyMLTools.metrics import accuracy_score\n",
    "print(accuracy_score(y_test, y_predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b174c2b-46d2-4765-96db-27b6e34ddfeb",
   "metadata": {},
   "source": [
    "**（6）将分类准确度添加进自定义knn中**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2e99fce-7152-49e0-b6af-fee952767e8d",
   "metadata": {},
   "source": [
    "> [MyMLTools/neighbors.py](MyMLTools/neighbors.py)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4aaaaa4a-a4a0-4ee4-aaf8-98c899741834",
   "metadata": {},
   "source": [
    "**（7）使用自定义knn中的分类准确度**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d4559f2c-c56c-4788-bbf9-5e344d10a728",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9860724233983287\n"
     ]
    }
   ],
   "source": [
    "from MyMLTools.neighbors import KNeighborsClassifier\n",
    "knn = KNeighborsClassifier(k=6)\n",
    "knn.fit(X_train, y_train)\n",
    "print(knn.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab9b8017-aa44-4b5a-9c30-635af89893b5",
   "metadata": {},
   "source": [
    "**（8）使用sklean中的相关内容**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "0de74186-8240-4a94-9aa4-294f23384c49",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9888888888888889\n",
      "0.9888888888888889\n"
     ]
    }
   ],
   "source": [
    "# 先切割数据\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2, random_state=666)\n",
    "# 使用knn算法\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "knn = KNeighborsClassifier(n_neighbors=6)\n",
    "knn.fit(X_train, y_train)\n",
    "y_predict = knn.predict(X_test)\n",
    "# 计算准确度\n",
    "from sklearn.metrics import accuracy_score\n",
    "print(accuracy_score(y_predict, y_test))\n",
    "print(knn.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adc4e199-371f-4ce0-8cfa-bf8040698bb1",
   "metadata": {},
   "source": [
    "## 1.7 超参数与模型参数"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "803ae255-6fa1-4d82-8305-7e14271ac8f2",
   "metadata": {},
   "source": [
    "超参数：在算法运行前需要决定的参数\n",
    "\n",
    "模型参数：算法过程中学习的参数\n",
    "\n",
    "KNN算法没有模型参数，但是有一些超参数"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0bda9b18-cf97-4eaa-b243-4300e2e9fcee",
   "metadata": {},
   "source": [
    "**（1）准备数据**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "f2fb5bd2-1286-4c81-8add-0377884e9722",
   "metadata": {},
   "outputs": [],
   "source": [
    "digits = datasets.load_digits()\n",
    "X = digits.data\n",
    "y = digits.target\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2, random_state=123)\n",
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c543639-437e-4f7b-a93c-d12608fe3e4b",
   "metadata": {},
   "source": [
    "**（2）超参数：近邻数量 k**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ab39daf8-a7d2-4aee-9879-372cc77b5694",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "best_k = 1\n",
      "best_score = 0.9861111111111112\n"
     ]
    }
   ],
   "source": [
    "best_score = 0.0\n",
    "best_k = 0\n",
    "for k in range(1,11):\n",
    "    knn = KNeighborsClassifier(n_neighbors=k)\n",
    "    knn.fit(X_train, y_train)\n",
    "    score = knn.score(X_test, y_test)\n",
    "    if score > best_score:\n",
    "        best_score = score\n",
    "        best_k = k\n",
    "print(f\"best_k = {best_k}\")\n",
    "print(f'best_score = {best_score}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81ee4aa2-a4a8-4c61-bb58-7fcf10a26712",
   "metadata": {},
   "source": [
    "**（3）超参数：距离权重 weights**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3acacf74-cedd-4105-9731-8d1486096bf4",
   "metadata": {},
   "source": [
    "**等权重策略 uniform**\n",
    "\n",
    "所有 K 个最近邻样本的投票权重完全相同，最终预测结果由 “多数表决”（分类任务）或 “简单平均”（回归任务）决定。\n",
    "\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250823095756412.png\" alt=\"image-20250823095756412\" style=\"zoom:40%;\" />\n",
    "\n",
    "**距离加权策略 distance**\n",
    "\n",
    "邻居的权重与其到目标样本的 “距离成反比”—— 距离越近的样本，权重越大；距离越远的样本，权重越小。\n",
    "\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250823095840509.png\" alt=\"image-20250823095840509\" style=\"zoom:40%;\" />\n",
    "\n",
    "同时也能更好的解决平票的问题。\n",
    "\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250823100328712.png\" alt=\"image-20250823100328712\" style=\"zoom:38%;\" />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "c241edac-036c-4b96-9a7b-3a9e60fb8182",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "当前计算过程：k = 1, method = uniform, score = 0.9861111111111112\n",
      "当前计算过程：k = 2, method = uniform, score = 0.9805555555555555\n",
      "当前计算过程：k = 3, method = uniform, score = 0.9861111111111112\n",
      "当前计算过程：k = 4, method = uniform, score = 0.9861111111111112\n",
      "当前计算过程：k = 5, method = uniform, score = 0.9805555555555555\n",
      "当前计算过程：k = 6, method = uniform, score = 0.9777777777777777\n",
      "当前计算过程：k = 7, method = uniform, score = 0.9833333333333333\n",
      "当前计算过程：k = 8, method = uniform, score = 0.9861111111111112\n",
      "当前计算过程：k = 9, method = uniform, score = 0.9861111111111112\n",
      "当前计算过程：k = 10, method = uniform, score = 0.9861111111111112\n",
      "当前计算过程：k = 1, method = distance, score = 0.9861111111111112\n",
      "当前计算过程：k = 2, method = distance, score = 0.9861111111111112\n",
      "当前计算过程：k = 3, method = distance, score = 0.9861111111111112\n",
      "当前计算过程：k = 4, method = distance, score = 0.9861111111111112\n",
      "当前计算过程：k = 5, method = distance, score = 0.9805555555555555\n",
      "当前计算过程：k = 6, method = distance, score = 0.9833333333333333\n",
      "当前计算过程：k = 7, method = distance, score = 0.9861111111111112\n",
      "当前计算过程：k = 8, method = distance, score = 0.9861111111111112\n",
      "当前计算过程：k = 9, method = distance, score = 0.9861111111111112\n",
      "当前计算过程：k = 10, method = distance, score = 0.9861111111111112\n",
      "best_score = 0.9861111111111112\n",
      "best_k = 1\n",
      "best_method = uniform\n"
     ]
    }
   ],
   "source": [
    "best_score = 0.0\n",
    "best_k = 0\n",
    "best_method = \"\"\n",
    "for method in ['uniform', 'distance']:\n",
    "    for k in range(1,11):\n",
    "        knn = KNeighborsClassifier(n_neighbors=k, weights=method)\n",
    "        knn.fit(X_train, y_train)\n",
    "        score = knn.score(X_test,y_test)\n",
    "        print(f\"当前计算过程：k = {k}, method = {method}, score = {score}\")\n",
    "        if score > best_score:\n",
    "            best_score = score\n",
    "            best_k = k\n",
    "            best_method = method\n",
    "print(f'best_score = {best_score}')\n",
    "print(f'best_k = {best_k}')\n",
    "print(f\"best_method = {best_method}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e22f8dd9-ebb5-43a5-af11-1066d2cd209e",
   "metadata": {},
   "source": [
    "**（4）超参数：明可夫斯基距离 p**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57a9be41-b27d-4b8a-96e5-ddfbda8b0711",
   "metadata": {},
   "source": [
    "先讨论欧拉距离和曼哈顿距离\n",
    "\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250823104456562.png\" alt=\"image-20250823104456562\" style=\"zoom:25%;\" />\n",
    "\n",
    "欧拉距离的本质是 “两点在空间中各维度坐标差值的平方和的平方根”，其公式随空间维度不同略有变化，但逻辑一致。\n",
    "\n",
    "$$\n",
    "\\sqrt{\\sum^n_{i=1}{\\left(X^{(a)}_i-X^{(b)}_i\\right)^2}}\n",
    "$$\n",
    "\n",
    "曼哈顿距离的本质是 “两点在空间中各维度坐标差值的绝对值之和”，不涉及平方和开根号，计算逻辑与欧拉距离有显著区别，但同样可扩展到任意 n 维空间。\n",
    "$$\n",
    "\\sum^n_{i=1}{\\left|X^{(a)}_i-X^{(b)}_i\\right|}\n",
    "$$\n",
    "\n",
    "显然，两个公式还是非常相像的，现在对两个公式进行变形推导：\n",
    "\n",
    " - 欧拉公式推导：\n",
    "$$\\sqrt{\\sum^n_{i=1}{(X^{(a)}_i-X^{(b)}_i)^2}}\\quad\\Rightarrow\\quad\\left(\\sum^n_{i=1}{\\left|X^{(a)}_i-X^{(b)}_i\\right|^2}\\right)^\\frac{1}{2}$$\n",
    "\n",
    " - 曼哈顿公式推导：\n",
    "$$\\sum^n_{i=1}{\\left|X^{(a)}_i-X^{(b)}_i\\right|}\\quad\\Rightarrow\\quad\\left(\\sum^n_{i=1}{\\left|X^{(a)}_i-X^{(b)}_i\\right|^1}\\right)^\\frac{1}{1}$$\n",
    "\n",
    "\n",
    "综合上述公式推导，得出 明可夫斯基距离 及 超参数 p：\n",
    "$$\n",
    "\\left(\\sum^n_{i=1}{\\left|X^{(a)}_i-X^{(b)}_i\\right|^p}\\right)^\\frac{1}{p}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "e437485e-ebe4-4c82-a67c-81bdefa5b5f6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "best_score = 0.9888888888888889\n",
      "best_k = 1\n",
      "best_p = 7\n"
     ]
    }
   ],
   "source": [
    "best_p = 0\n",
    "best_score = 0.0\n",
    "best_k = 0\n",
    "for k in range(1,11):\n",
    "    for p in range(1,11):\n",
    "        knn = KNeighborsClassifier(n_neighbors=k,p=p,weights='distance')\n",
    "        knn.fit(X_train, y_train)\n",
    "        score = knn.score(X_test, y_test)\n",
    "        if score > best_score:\n",
    "            best_k = k\n",
    "            best_p = p\n",
    "            best_score = score\n",
    "print(f'best_score = {best_score}')\n",
    "print(f'best_k = {best_k}')\n",
    "print(f\"best_p = {best_p}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "a5bcc5b0-da26-4793-a4a9-180ab0ba4ef3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "best_score = 0.9888888888888889\n",
      "best_k = 1\n",
      "best_p = 7\n"
     ]
    }
   ],
   "source": [
    "best_p = 0\n",
    "best_score = 0.0\n",
    "best_k = 0\n",
    "for k in range(1,11):\n",
    "    for p in range(1,11):\n",
    "        knn = KNeighborsClassifier(n_neighbors=k,p=p,weights='uniform')\n",
    "        knn.fit(X_train, y_train)\n",
    "        score = knn.score(X_test, y_test)\n",
    "        if score > best_score:\n",
    "            best_k = k\n",
    "            best_p = p\n",
    "            best_score = score\n",
    "print(f'best_score = {best_score}')\n",
    "print(f'best_k = {best_k}')\n",
    "print(f\"best_p = {best_p}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f423be5-23bc-4e1d-8791-2f56d6459634",
   "metadata": {},
   "source": [
    "## 1.8 网格搜索超参数"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55f89c76-6a21-4965-9a0a-d004c75ceebd",
   "metadata": {},
   "source": [
    "网格搜索是一种最常用的超参数优化方法，核心思想是通过 “穷举遍历” 所有可能的超参数组合，找到使模型性能最优的组合。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08645380-f5a3-432a-8b40-9f34587a4a1e",
   "metadata": {},
   "source": [
    "**（1）创建超参数空间**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "1374be42-efbb-4c7a-8c83-10f47e2b8301",
   "metadata": {},
   "outputs": [],
   "source": [
    "param_grid = [\n",
    "    {\n",
    "        'weights':['uniform'],\n",
    "        'n_neighbors':[i for i in range(1,11)]\n",
    "    },\n",
    "    {\n",
    "        'weights':['distance'],\n",
    "        'n_neighbors':[i for i in range(1,11)],\n",
    "        'p': [i for i in range(1,6)]\n",
    "    }\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d19a3ad6-c957-44b2-933c-b7bc2715a446",
   "metadata": {},
   "source": [
    "**（2）使用GridSearchCV**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "8cf26ae0-08de-4093-98bd-8648efcf55c6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: #000;\n",
       "  --sklearn-color-text-muted: #666;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-1 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-1 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: flex;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "  align-items: start;\n",
       "  justify-content: space-between;\n",
       "  gap: 0.5em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label .caption {\n",
       "  font-size: 0.6rem;\n",
       "  font-weight: lighter;\n",
       "  color: var(--sklearn-color-text-muted);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-1 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-1 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 0.5em;\n",
       "  text-align: center;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-1 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(estimator=KNeighborsClassifier(),\n",
       "             param_grid=[{&#x27;n_neighbors&#x27;: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n",
       "                          &#x27;weights&#x27;: [&#x27;uniform&#x27;]},\n",
       "                         {&#x27;n_neighbors&#x27;: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n",
       "                          &#x27;p&#x27;: [1, 2, 3, 4, 5], &#x27;weights&#x27;: [&#x27;distance&#x27;]}])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>GridSearchCV</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.model_selection.GridSearchCV.html\">?<span>Documentation for GridSearchCV</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></div></label><div class=\"sk-toggleable__content fitted\"><pre>GridSearchCV(estimator=KNeighborsClassifier(),\n",
       "             param_grid=[{&#x27;n_neighbors&#x27;: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n",
       "                          &#x27;weights&#x27;: [&#x27;uniform&#x27;]},\n",
       "                         {&#x27;n_neighbors&#x27;: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n",
       "                          &#x27;p&#x27;: [1, 2, 3, 4, 5], &#x27;weights&#x27;: [&#x27;distance&#x27;]}])</pre></div> </div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>best_estimator_: KNeighborsClassifier</div></div></label><div class=\"sk-toggleable__content fitted\"><pre>KNeighborsClassifier(n_neighbors=2, weights=&#x27;distance&#x27;)</pre></div> </div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>KNeighborsClassifier</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.neighbors.KNeighborsClassifier.html\">?<span>Documentation for KNeighborsClassifier</span></a></div></label><div class=\"sk-toggleable__content fitted\"><pre>KNeighborsClassifier(n_neighbors=2, weights=&#x27;distance&#x27;)</pre></div> </div></div></div></div></div></div></div></div></div>"
      ],
      "text/plain": [
       "GridSearchCV(estimator=KNeighborsClassifier(),\n",
       "             param_grid=[{'n_neighbors': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n",
       "                          'weights': ['uniform']},\n",
       "                         {'n_neighbors': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n",
       "                          'p': [1, 2, 3, 4, 5], 'weights': ['distance']}])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn = KNeighborsClassifier()\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "grid_search = GridSearchCV(knn, param_grid)\n",
    "grid_search.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "6649d912-76d4-4472-aec6-bd932b76246b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KNeighborsClassifier(n_neighbors=2, weights='distance')\n"
     ]
    }
   ],
   "source": [
    "print(grid_search.best_estimator_) # 返回的是携带了最好超参数的算法对象"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "d0dd2776-29ab-45d0-98cf-28b60a061440",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9895639759969027\n"
     ]
    }
   ],
   "source": [
    "print(grid_search.best_score_) # 最高得分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "a85df703-3383-4bb3-bd26-0bce01a125fa",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'n_neighbors': 2, 'p': 2, 'weights': 'distance'}\n"
     ]
    }
   ],
   "source": [
    "print(grid_search.best_params_)# 最好参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "d6eefd30-bc5b-4e6f-a3b5-2c7338287301",
   "metadata": {},
   "outputs": [],
   "source": [
    "knn = grid_search.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "fc994c3c-d755-4107-a12e-7f319e50d36c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9861111111111112\n"
     ]
    }
   ],
   "source": [
    "print(knn.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4d82845-d722-4347-96d6-fc999bfa490d",
   "metadata": {},
   "source": [
    "## 1.9 数据归一化处理"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec626192-1d61-4b48-8c47-84f7554f7a7a",
   "metadata": {},
   "source": [
    "数据归一化是一种核心的数据预处理技术，其目的是将不同特征（或变量）的取值范围映射到统一、标准化的区间内，消除因特征量纲（单位）或数值尺度差异对模型训练和分析结果的干扰。\n",
    "\n",
    "如果不同特征的取值范围差异过大，会导致模型 “偏向” 数值更大的特征，忽略数值较小但重要的特征，最终影响模型性能。\n",
    "\n",
    "|       | 肿瘤大小（厘米） | 发现时间（天） |\n",
    "| ----- | ---------------- | -------------- |\n",
    "| 样本1 | 1                | 200            |\n",
    "| 样本2 | 5                | 100            |\n",
    "\n",
    "两个样本的距离：$\\sqrt{(1-5)^2 + (200-100)^2} = \\sqrt{16+10000}$\n",
    "\n",
    "- **发现时间**的数值差异会完全主导距离计算\n",
    "- **肿瘤大小**的影响被严重削弱\n",
    "\n",
    "|       | 肿瘤大小（厘米） | 发现时间（年） |\n",
    "| ----- | ---------------- | -------------- |\n",
    "| 样本1 | 1                | 0.55           |\n",
    "| 样本2 | 5                | 0.27           |\n",
    "\n",
    "两个样本的距离：$\\sqrt{(1-5)^2 + (0.55-0.27)^2} = \\sqrt{16+0.078}$\n",
    "\n",
    "- **肿瘤大小**的数值差异会完全主导距离计算\n",
    "- **发现时间**的影响被严重削弱。\n",
    "\n",
    "都会导致模型无法正确学习到特征信息。\n",
    "\n",
    "**解决办法1：最值归一化**\n",
    "\n",
    "将所有的数据映射到统一尺度，最值归一化的目的就是把数据映射到0-1之间\n",
    "$$\n",
    "x_{scale} = \\frac{x-x_{min}}{x_{max}-x_{min}}\n",
    "$$\n",
    "\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250823130002573.png\" alt=\"image-20250823130002573\" style=\"zoom:50%;\" />\n",
    "\n",
    "优点：\n",
    "\n",
    "- 线性变换过程透明，结果可解释性强；\n",
    "- 可自定义映射区间，满足不同算法需求，$[a,b]\\quad ,x_{scale} = a+\\frac{(x-x_{min})×(b-a)}{x_{max}-x_{min}}$；\n",
    "- 原始数据的相对大小关系不会改变\n",
    "\n",
    "缺点：\n",
    "\n",
    "- 若数据中存在极端值，会严重拉大极差，导致大部分正常数据被压缩到极小范围；\n",
    "- 必须先知道特征的最大值和最小值\n",
    "\n",
    "**解决办法2：均值方差归一化**\n",
    "\n",
    "把所有数据归一到均值为0方差为1的分布中（通常为[−3,3]），均值方差归一化通过均值和标准差计算，受极端异常值的影响更小\n",
    "$$\n",
    "x_{scale}=\\frac{x-x_{mean}}{S}\n",
    "$$\n",
    "<img src=\"https://zh-1258460726.cos.ap-nanjing.myqcloud.com//typora-pic/image-20250823130249235.png\" alt=\"image-20250823130249235\" style=\"zoom:50%;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c44b2722-7425-4290-80cc-3ec28bd69022",
   "metadata": {},
   "source": [
    "**（1）最值归一化**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "49de9399-8a94-407a-aca7-63539046042c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAKnRJREFUeJzt3X9s1Pdh//HX2RAbMvsaE9l3DiQ1ERJ13S5ASr4hqLC0pqjUSxWtW0lZk1WKEiekuJEawtLNuFrswDYUrbTugqYolUepqqZNXG0sbsncMsjMcGgxnpKOeoQRn7wBvXMHhubu/f3Duytn3+H7+bn353PPh+Q/+NyH4837Pv58Xvf+6TPGGAEAADikotQFAAAA5YXwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABw1LxSF2CmWCymd999VzU1NfL5fKUuDgAAyIAxRpOTk2psbFRFxfXbNqwLH++++66WLFlS6mIAAIAcnD17VosXL77uOdaFj5qaGknTha+trS1xaQAAQCYikYiWLFmSeI5fj3XhI97VUltbS/gAAMBlMhkywYBTAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOInwAAABHET4AAICjCB8AAMBR1i0yBgC2iMaMhsYuaGJySvU11VrdVKfKCvacAvJF+ACAFA6OjKurf1Tj4anEsaC/Wp1tzdrYEixhyQD3o9sFAGY4ODKu9r7hpOAhSaHwlNr7hnVwZLxEJQO8gfABANeIxoy6+kdlUrwWP9bVP6poLNUZADJB+ACAawyNXZjV4nEtI2k8PKWhsQvOFQrwGMIHAFxjYjJ98MjlPACzET4A4Br1NdUFPQ/AbIQPALjG6qY6Bf3VSjeh1qfpWS+rm+qcLBbgKYQPFEQ0ZnT09Hm9cuKcjp4+z2A8uFZlhU+dbc2SNCuAxP/c2dbMeh9AHljnA3ljPQR4zcaWoHq3rJx1XQe4roGC8BljrPqKGolE5Pf7FQ6HVVtbW+riYA7x9RBmXkTx74S9W1Zyo4ZrscIpkLlsnt+0fCBnc62H4NP0egitzQFu2HClygqf7r59UamLAXgOYz6QM9ZDAADkgvCBnLEeAgAgF4QP5Iz1EAAAuSibMR8MHCu8+HoIofBUynEfPk3PDmA9BHgF9xGgMMoifDAVtDji6yG09w3LJyUFENZDgNdwHwEKx/PdLmyNXVzx9RAC/uSulYC/mmm28AzuI0BheXqdj2jMaO2uQ2lnZMS7BQ5vv5dv53miORpexX0EyAzrfPyfbKaCMpc/P6yHAK/iPgIUnqe7XZgKCiBf3EeAwvN0+GAqKIB8cR8BCs/T4YOtsQHki/sIUHieDh9sjQ0gX9xHgMLzdPiQmAoKIH/cR4DC8vRU22sxFRRAvriPAOkx1TYFpoICyBf3EaAwyiZ88I0FAAA7lEX4YE8GAADs4fkBp+zJAACAXTwdPqIxo67+0ZTbvcePdfWPKhqzaswtAACe5unwkc2eDAAAwBmeDh/syQAAgH08HT7YkwEAAPt4erZLfE+GUHgq5bgPn6ZXKGRPBuSC6dsAkBtPh4/4ngztfcPySUkBhD0ZkA+mbwNA7jzd7SKxJwMKj+nbAJAfT7d8xG1sCaq1OUATOfI21/Rtn6anb7c2B7i+ACCNsggfEnsyoDCymb7N9QYAqXm+2wUoJKZvA0D+CB9AFpi+DQD5I3wAWYhP3043msOn6VkvTN8GgPQIH0AW4tO3Jc0KIEzfLr5ozOjo6fN65cQ5HT19nn2ZAJcqmwGnQKHEp2/PXOcjwDofRcXaKoB3+IwxVn11iEQi8vv9CofDqq2tLXVxgLRY4dQ58bVVZt6s4rXNmj1A6WXz/KblA8gR07edwdoq8AobvrDYUAaJ8AHAcqytAi+wodvQhjLEMeAUgNVYWwVuZ8OWDDaU4VqEDwBWY20VuNlc3YbSdLdhMWdu2VCGmQgfAKzG2ipws2y6Db1chpkIHwCsxtoqcDMbug1tKMNMhA8A1ouvrRLwJ3etBPzVTLOF1WzoNrShDDMx2wWAK2xsCaq1OWDFNEEgU/Fuw1B4KuWYC5+mQ3Qxuw1tKMNMtHwAcI342ir33XGL7r59EcED1rOh29CGMsxE+AAAoIhs6Da0oQzXYnl1AAAcYMPqosUsA8urAwBgGRu2ZLChDFKW3S7vvfeevvKVr6ipqUkLFizQ0qVL9dWvflWxWCxxjjFGO3fuVGNjoxYsWKD169fr1KlTBS84AABwp6zCx65du/TNb35Te/fu1b//+79r9+7d+su//Et97WtfS5yze/du7dmzR3v37tWxY8cUCATU2tqqycnJghceAAC4T1bh4+jRo7rvvvu0adMmvf/979cf/MEfaMOGDfq3f/s3SdOtHs8//7yeeeYZ3X///WppadFLL72kS5cuaf/+/UX5DwAAAHfJKnysXbtWP/7xj/X2229Lkn72s5/p8OHD+uQnPylJGhsbUygU0oYNGxJ/p6qqSuvWrdORI0dSvueVK1cUiUSSfoC5RGNGR0+f1ysnzuno6fOO7kkAAMhPVgNOt2/frnA4rOXLl6uyslLRaFTPPvusNm/eLEkKhUKSpIaGhqS/19DQoDNnzqR8z56eHnV1deVSdpQpm7aFBgBkL6uWj+985zvq6+vT/v37NTw8rJdeekl/9Vd/pZdeeinpPJ8vedqOMWbWsbgdO3YoHA4nfs6ePZvlfwHlxLZtoQEA2cuq5ePLX/6ynn76aX32s5+VJH3oQx/SmTNn1NPTowcffFCBQEDSdAtIMPjbb6ATExOzWkPiqqqqVFVVlWv5UUbm2hbap+ltoVubA6x8CQAWy6rl49KlS6qoSP4rlZWViam2TU1NCgQCGhgYSLx+9epVDQ4Oas2aNQUoLsqZjdtCAwCyl1XLR1tbm5599lndeuut+uAHP6g333xTe/bs0Re+8AVJ090tHR0d6u7u1rJly7Rs2TJ1d3dr4cKFeuCBB4ryH0D5sHFbaABA9rIKH1/72tf0Z3/2Z3rsscc0MTGhxsZGPfLII/rzP//zxDlPPfWULl++rMcee0wXL17UXXfdpddee001NTUFLzzKi43bQgMAssfeLnCNaMxo7a5Dc24LfXj7vYz5AACHZfP8ZldbuIaN20IDALJH+ICr2LYtNAAge+xqC9fZ2BJUa3Og5FtTAwByQ/iAK9myLTQA50Vjhi8fLkf4AAC4BtsreANjPgAArsD2Ct5B+AAAWG+u7RWk6e0V2OHaHQgfAADrsb2CtxA+AADWY3sFbyF8AACsx/YK3kL4AABYb3VTnYL+6lmrG8f5ND3rZXVTnZPFQo4IHwAA67G9grcQPgAArsD2Ct7BImMAANdgewVvIHwAAFyF7RXcj/ABAB7D3iewHeEDADyEvU/gBgw4BQCPYO8TuAXhAwA8gL1P4CaEDwDwAPY+gZsQPgDAA9j7BG5C+AAAD2DvE7gJs10swxQ5ALmI730SCk+lHPfh0/RKoOx9AhsQPizCFDkAuYrvfdLeNyyflBRA2PsEtqHbxRJMkQOQL/Y+gVvQ8mGBuabI+TQ9Ra61OcC3FgDXxd4ncAPChwWymSLHfgYA5sLeJ7Ad3S4WYIocAKCcED4swBQ5AEA5IXxYID5FLl2PrE/Ts16YIgcA8ALChwXiU+QkzQogTk+Ri8aMjp4+r1dOnNPR0+fZB6JE+BwAe/H7mT8GnFoiPkVu5jofAQfX+WCdETvwOQD24vezMHzGGKsiWyQSkd/vVzgcVm1tbamL47hSrXAaX2dk5sUQ/5dZI8AZfA6Avfj9vL5snt90u1gmPkXuvjtu0d23L3Ksq4WtuEuPzwGwF7+fhUX4AFtxW4LPAbAXv5+FRfgA64xYgs8BsBe/n4VF+ADrjFiCzwGwF7+fhUX4AOuMWILPAbAXv5+FRfiAVeuMlDM+B8Be/H4WFuEDktiK2xZ8DoC9+P0sHNb5QJJSrTOCZHwOgL34/Uwtm+c3K5wiCVtx24HPAbAXv5/5o9sFAAA4ivABAAAcRfgAAACOInwAAABHET4AAICjCB8AAMBRTLUFMlDMef2sGQDgWuVwTyB8AHM4ODKurv7RpO20g/5qdbY1572iYTHfG4D7lMs9gW4X4DoOjoyrvW846UYgSaHwlNr7hnVwZNzK9wbgPuV0TyB8AGlEY0Zd/aNKtf9A/FhX/6iisex3KCjmewNwn3K7JxA+YJ1ozOjo6fN65cQ5HT19vmS/bENjF2Z9A7mWkTQentLQ2AWr3huA+xTinmDLvTMTjPmAVWzq75yYTH8jyOU8p94bgPvke0+w6d6ZCVo+YA3b+jvra6rnPimL85x6bwDuk889wbZ7ZyYIH7CCjf2dq5vqFPRXK90EN5+mv1msbqqz6r0BuE+u9wQb752ZIHzACk6NgcimT7SywqfOtmZJmnVDiP+5s605p/n3xXxvAO6T6z3BrePHCB+wghNjIA6OjGvtrkPavO8NbTtwQpv3vaG1uw5dt0lyY0tQvVtWKuBPbuoM+KvVu2VlXn2pxXxvAO6Tyz3BrePHGHAKKxR7DES8T3RmO0e8T/R6D/uNLUG1NgeKsuJgMd8bgPtke09w6/gxwgesEO/vDIWnUvZd+jSd/nMZAzFXn6hP032irc2BtL/glRU+3X37oqz/7UwU870BuE8294Ri3juLiW4XWKGYYyDc2icKAHNx6/gxwgesUawxEG7tEwWATLhx/BjdLrBKMcZAuLVPFAAy5bbxY4QPWKfQYyDc2icKANlw0/ixrLtdzp07py1btmjRokVauHCh7rjjDh0/fjzxujFGO3fuVGNjoxYsWKD169fr1KlTBS00kA239okCgFdlFT4uXryoe+65R/Pnz9c//uM/anR0VH/913+t973vfYlzdu/erT179mjv3r06duyYAoGAWltbNTk5WeiyAxlzY58oAHiVzxiT8ZqrTz/9tP7lX/5FP/3pT1O+boxRY2OjOjo6tH37dknSlStX1NDQoF27dumRRx6Z89+IRCLy+/0Kh8Oqra3NtGhARqIx45o+UQBwk2ye31m1fLz66qu688479ZnPfEb19fVasWKF9u3bl3h9bGxMoVBIGzZsSByrqqrSunXrdOTIkZTveeXKFUUikaQfoFjifaL33XGL7r59EcEDAEogq/Dxy1/+Ur29vVq2bJn+6Z/+SY8++qi++MUv6lvf+pYkKRQKSZIaGhqS/l5DQ0PitZl6enrk9/sTP0uWLMnl/wEAAFwiq/ARi8W0cuVKdXd3a8WKFXrkkUf08MMPq7e3N+k8ny/526QxZtaxuB07digcDid+zp49m+V/AQAAuElW4SMYDKq5uTnp2Ac+8AG98847kqRAICBJs1o5JiYmZrWGxFVVVam2tjbpBwAAeFdW4eOee+7RW2+9lXTs7bff1m233SZJampqUiAQ0MDAQOL1q1evanBwUGvWrClAcQHAOdGY0dHT5/XKiXM6evq8orGMx+cDuI6sFhn70pe+pDVr1qi7u1t/+Id/qKGhIb3wwgt64YUXJE13t3R0dKi7u1vLli3TsmXL1N3drYULF+qBBx4oyn8AAIrh4Mi4uvpHk/YFCvqr1dnWzNRsIE9ZTbWVpB/+8IfasWOHfvGLX6ipqUlPPvmkHn744cTrxhh1dXXpb//2b3Xx4kXddddd+vrXv66WlpaM3p+ptgBK7eDIuNr7hmetiBsfucbaMMBs2Ty/sw4fxUb4AFBK0ZjR2l2H0u6EHF+O//D2e5mqXQSsxeNe2Ty/2dsFAK4xNHYhbfCQJCNpPDylobELrtlHwy3o6iofWe/tAgBeNjGZPnjkch4yE+/qmhn8QuEptfcN6+DIeIlKhmIgfADANeprquc+KYvzMLdozKirfzTlrtPxY139o8w28hDCBwBcY3VTnYL+6lk7IMf5NN0VsLqpzslieVo2XV3wBsIHAFyjssKnzrbpxRRnBpD4nzvbmhkEWUB0dZUfwgcAzLCxJajeLSsV8Cd3rQT81UyzLQK6usoPs10AIIWNLUG1NgeY9umAeFdXKDyVctxHfHozXV3eQfgAgDQqK3xMp3VAvKurvW9YPikpgNDV5U10uwAASo6urvJCywcAwAp0dZUPwgcAwBp0dZUHul0AAICjCB8AAMBRhA8AAOAowgcAAHAU4QMAADiK8AEAABxF+AAAAI5inQ/kJBozLAQEAMgJ4QNZOzgyrq7+UY2Hf7u9ddBfrc62ZpZABgDMiW4XZOXgyLja+4aTgockhcJTau8b1sGR8RKVDADgFoQPZCwaM+rqH0255XX8WFf/qKKxVGcAADCNbhcHuX2cxNDYhVktHtcyksbDUxoau8DeDACQI7c/KzJB+HCIF8ZJTEymDx65nAcASOaFZ0Um6HZxgFfGSdTXVBf0PADAb3nlWZEJwkeReWmcxOqmOgX91UrX+OfTdEJf3VTnZLEAwPW89KzIBOGjyLIZJ2G7ygqfOtuaJWlWAIn/ubOt2XN9kwBQbF56VmSC8FFkXhsnsbElqN4tKxXwJ3etBPzV6t2y0lN9kgDgFK89K+bCgNMi8+I4iY0tQbU2Bzw/GhsAnOLFZ8X1ED6KLD5OIhSeStmX59N0q4HbxklUVviYTgsABeLVZ0U6dLsUGeMkAABzKbdnBeHDAYyTAADMpZyeFT5jjFXzdiKRiPx+v8LhsGpra0tdnIIqh1XrAAD5ceuzIpvnN2M+HMQ4CQDAXMrhWUG3CwAAcBThAwAAOIrwAQAAHMWYD8Al3DoIDQBmInwALlAu22wDKA90uwCWK6dttgGUB1o+ygjN9u4z1zbbPk1vs93aHOCzBDyiHO7VhI8yQbO9O2WzzbbX1wUAykG53KvpdikDNNu7V7ltsw2Us3K6VxM+PG6uZntputk+GrNqlX38n3LbZhsoV+V2ry6b8BGNGR09fV6vnDino6fPe+YDnEs2zfawT3yb7XS9vT5NN8nGt9ku1+sc9uFazE653avLYsxHufShpUKzvbvFt9lu7xuWT0r6VjRzm+1yvs5hF67F7JXbvdrzLR/l1IeWCs327pfJNtvlfp3DHlyLuSm3e7WnWz6YpvjbZvtQeCplPfg0/RCLN9vDThtbgmptDqScfsd1DltwLeau3O7Vnm75KLc+tFTizfaSZo0bmNlsD7vFt9m+745bdPftixKfGdc5bMG1mLtyu1d7OnyUWx9aOpk028O9uM5hC67F/JTTvdrT3S7l1od2Pddrtoe7cZ3DFlyL+SuXe7Wnw0e59aHNJd5sD2/hOoctuBYLoxzu1Z7udim3PjSUJ65z2IJrEZnydPiQyqsPDeWL6xy24FpEJnzGGKuWnYtEIvL7/QqHw6qtrS3Y+5bDLoEA1zlswbVYfrJ5fnt6zMe1yqEPDeA6hy24FnE9nu92AQAAdiF8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAclVf46Onpkc/nU0dHR+KYMUY7d+5UY2OjFixYoPXr1+vUqVP5lhMAAHhEzuHj2LFjeuGFF/ThD3846fju3bu1Z88e7d27V8eOHVMgEFBra6smJyfzLiwAAHC/nMLHr3/9a33uc5/Tvn37dNNNNyWOG2P0/PPP65lnntH999+vlpYWvfTSS7p06ZL2799fsEIDAAD3yil8PP7449q0aZM+/vGPJx0fGxtTKBTShg0bEseqqqq0bt06HTlyJOV7XblyRZFIJOkHAAB4V9Ybyx04cEDDw8M6duzYrNdCoZAkqaGhIel4Q0ODzpw5k/L9enp61NXVlW0xAACAS2XV8nH27Flt27ZNfX19qq6uTnuez5e8bbIxZtaxuB07digcDid+zp49m02RAACAy2TV8nH8+HFNTExo1apViWPRaFQ/+clPtHfvXr311luSpltAgsFg4pyJiYlZrSFxVVVVqqqqyqXsAABIkqIxo6GxC5qYnFJ9TbVWN9WpsiL1l16UXlbh42Mf+5hOnjyZdOxP/uRPtHz5cm3fvl1Lly5VIBDQwMCAVqxYIUm6evWqBgcHtWvXrsKVGgCA/3NwZFxd/aMaD08ljgX91epsa9bGluB1/iZKJavwUVNTo5aWlqRjN954oxYtWpQ43tHRoe7ubi1btkzLli1Td3e3Fi5cqAceeKBwpQYAQNPBo71vWGbG8VB4Su19w+rdspIAYqGsB5zO5amnntLly5f12GOP6eLFi7rrrrv02muvqaamptD/FACgjEVjRl39o7OChyQZST5JXf2jam0O0AVjGZ8xJtXnVjKRSER+v1/hcFi1tbWlLg4AwFJHT5/X5n1vzHnetx/+f7r79kUOlKi8ZfP8Zm8XAIArTUxOzX1SFufBOYQPAIAr1dekX/Ihl/PgHMIHAMCVVjfVKeivVrrRHD5Nz3pZ3VTnZLGQgYIPOEV5cOucereW21bUJ9Jx4tqorPCps61Z7X3D8klJA0/j/1JnWzPXpIUIH8iaW+fUu7XctqI+kY6T18bGlqB6t6yc9e8FuBatxmwXZCXdnPr49wpb59S7tdy2oj6RTqmuDVrhSo/ZLiiKuebUS9Nz6qMxq/Ksa8ttK+oT6ZTy2qis8Onu2xfpvjtu0d23LyJ4WI7wgYwNjV1IatacyUgaD09paOyCc4XKgFvLbSvqE+lwbSBThA9kzK1z6t1abltRn0iHawOZInwgY26dU+/WctuK+kQ6XBvIFOEDGXPrnHq3lttW1CfS4dpApggfyFh8Tr2kWTcXm+fUu7XctqI+kQ7XBjJF+EBW4nPqA/7kZtOAv9rq6ZVuLbetqE+kw7WBTLDOB3Li1jn1bi23rahPpMO1UX6yeX4TPgAAQN5YZAwAAFiL8AEAABxF+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAowgfAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOInwAAABHET4AAICj5pW6AEC5i8aMhsYuaGJySvU11VrdVKfKCl+piwUARUP4AEro4Mi4uvpHNR6eShwL+qvV2dasjS3BEpYMAIqHbhegRA6OjKu9bzgpeEhSKDyl9r5hHRwZL1HJAKC4CB9ACURjRl39ozIpXosf6+ofVTSW6gwAcDfCB1ACQ2MXZrV4XMtIGg9PaWjsgnOFAgCHED6AEpiYTB88cjkPANyE8AGUQH1NdUHPAwA3IXwAJbC6qU5Bf7XSTaj1aXrWy+qmOieLBQCOIHzMIRozOnr6vF45cU5HT59nACAKorLCp862ZkmaFUDif+5sa2a9DwCexDof18EaDCimjS1B9W5ZOesaC3CNAfA4nzHGqq/ykUhEfr9f4XBYtbW1JStHfA2GmZUT/x7au2UlDwcUBCucAvCCbJ7ftHykMNcaDD5Nr8HQ2hzgIYG8VVb4dPfti0pdDABwDGM+UijlGgyMMQEAeB0tHymUag0GxpgAAMoBLR8plGINBvb5AACUC8JHCk6vwcA+H4C30H0KXB/dLinE12Bo7xuWT0oKBcVYgyGbMSYMTATsRvcpMDdaPtKIr8EQ8Cd3rQT81QWfZss+H4A30H0KZIaWj+vY2BJUa3Og6GswsM8H4H5M0QcyR/iYgxNrMMTHmITCUylvXD5Nt7iwzwdgL7pPgczR7WIB9vkA3I/uUyBzhA9LODnGBEDh0X0KZI5uF4s4NcYEQOHRfQpkjvBhGfb5ANzJ6Sn6gJvR7QIABUL3KZAZWj4AoIDoPgXmRvgAgAJzsvs0GjMEHbhOVt0uPT09+shHPqKamhrV19fr05/+tN56662kc4wx2rlzpxobG7VgwQKtX79ep06dKmihAQDTK6qu3XVIm/e9oW0HTmjzvje0dtchVlKF9bIKH4ODg3r88cf1xhtvaGBgQO+99542bNig//3f/02cs3v3bu3Zs0d79+7VsWPHFAgE1NraqsnJyYIXHgDKFUu5w818xpict1v87//+b9XX12twcFAf/ehHZYxRY2OjOjo6tH37dknSlStX1NDQoF27dumRRx6Z8z0jkYj8fr/C4bBqa2tzLRoAeFY0ZrR216G0K6rGp/Ue3n4vXTBwTDbP77xmu4TDYUlSXd30vPWxsTGFQiFt2LAhcU5VVZXWrVunI0eOpHyPK1euKBKJJP0AsA/bxNsjm6XcARvlPODUGKMnn3xSa9euVUtLiyQpFApJkhoaGpLObWho0JkzZ1K+T09Pj7q6unItBgAHsE28XVjKHW6Xc8vH1q1b9fOf/1zf/va3Z73m8yU38xljZh2L27Fjh8LhcOLn7NmzuRYJQBEwtsA+LOUOt8spfDzxxBN69dVX9frrr2vx4sWJ44FAQNJvW0DiJiYmZrWGxFVVVam2tjbpB4Ad5tomXpreJp4uGGfFl3JPN5rDp+mWKZZyh62yCh/GGG3dulUvv/yyDh06pKampqTXm5qaFAgENDAwkDh29epVDQ4Oas2aNYUpMQDHMLbATuyEDbfLKnw8/vjj6uvr0/79+1VTU6NQKKRQKKTLly9Lmu5u6ejoUHd3t77//e9rZGREDz30kBYuXKgHHnigKP8BAMXD2AJ7sZQ73CyrAae9vb2SpPXr1ycdf/HFF/XQQw9Jkp566ildvnxZjz32mC5evKi77rpLr732mmpqagpSYADOYWyB3VjKHW6V1zofxcA6H4A94utJzLVNPOtJAHBsnQ8A3sbYAgDFQPgAcF2MLQBQaOxqC2BOjC0AUEi0fAAAAEfR8gFgTiyvDqCQaPkAcF0srw6g0AgfANJieXUAxUD4AJAWy6sDKAbCB4C0WF4dQDEQPgCkxfLqAIqB8AEgLbZuB1AMhA+4UjRmdPT0eb1y4pyOnj6vaMykPIb8sLw6gGJgnQ+4Tqo1J963cL4k6VeXfpM4xjoUhRFfXn1mnQeoXwA5YldbuEp8zYlMLtr4d3H2HymMaMywvDqAtLJ5ftPyAde43poTqRhNB5Cu/lG1Ngd4UOapssKnu29fVOpiAPAAxnzANeZacyIV1qEA7MU4rfJFywdcI5+1JFiHArAL+wWVN1o+4Br5rCXBOhSAPdgvCISPa9AEaLe51pxIhXUoALuwXxAkul0SaAK0X3zNifa+YfmkOQeesg4FYJ9s9gtigLN30fIhmgDdJL7mRMCf3I1y08L5ibU+4gL+aqbZApZhvyBItHzM2QTIVE37bGwJqrU5MGvNCUmsQwFYjv2CIBE+aAJ0qXRrTvAZAXaLj90KhadSfunzabrVknFa3lb23S40AQKAc9gvCBLhgyZAAHBYurFbjNMqH2Xf7UITIAA4L93YLVo8ykPZh4/rTd+kCRAAiof9gspX2Xe7SDQBAgDgpLJv+YijCRAAAGcQPq5BEyAAAMVHtwsAAHAU4QMAADiK8AEAABxF+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcBThAwAAOIrwAQAAHEX4AAAAjiJ8AAAARxE+AACAo+aVugBAJqIxo6GxC5qYnFJ9TbVWN9WpssJX6mIBAHJA+ID1Do6Mq6t/VOPhqcSxoL9anW3N2tgSLGHJAAC5oNsFVjs4Mq72vuGk4CFJofCU2vuGdXBkvEQlAwDkivABa0VjRl39ozIpXosf6+ofVTSW6gwAgK0IH7DW0NiFWS0e1zKSxsNTGhq74FyhAAB5I3zAWhOT6YNHLucBAOxA+IC16muqC3oeAMAOhA9Ya3VTnYL+aqWbUOvT9KyX1U11ThYLAJAnwgesVVnhU2dbsyTNCiDxP3e2NbPeBwC4DOEDVtvYElTvlpUK+JO7VgL+avVuWck6HwDgQiwyButtbAmqtTnACqcA4BGED7hCZYVPd9++qNTFAAAUAN0uAADAUYQPAADgKMIHAABwFOEDAAA4ivABAAAcRfgAAACOInwAAABHET4AAICjCB8AAMBR1q1waoyRJEUikRKXBAAAZCr+3I4/x6/HuvAxOTkpSVqyZEmJSwIAALI1OTkpv99/3XN8JpOI4qBYLKZ3331XNTU18vky3zgsEoloyZIlOnv2rGpra4tYQsRR586jzp1HnTuPOndeIercGKPJyUk1NjaqouL6ozqsa/moqKjQ4sWLc/77tbW1XKwOo86dR507jzp3HnXuvHzrfK4WjzgGnAIAAEcRPgAAgKM8Ez6qqqrU2dmpqqqqUhelbFDnzqPOnUedO486d57TdW7dgFMAAOBtnmn5AAAA7kD4AAAAjiJ8AAAARxE+AACAozwRPr7xjW+oqalJ1dXVWrVqlX7605+Wukie0dPTo4985COqqalRfX29Pv3pT+utt95KOscYo507d6qxsVELFizQ+vXrderUqRKV2Ft6enrk8/nU0dGROEZ9F8e5c+e0ZcsWLVq0SAsXLtQdd9yh48ePJ16n3gvrvffe01e+8hU1NTVpwYIFWrp0qb761a8qFoslzqHO8/OTn/xEbW1tamxslM/n0w9+8IOk1zOp3ytXruiJJ57QzTffrBtvvFG///u/r//6r//Kv3DG5Q4cOGDmz59v9u3bZ0ZHR822bdvMjTfeaM6cOVPqonnCJz7xCfPiiy+akZERc+LECbNp0yZz6623ml//+teJc5577jlTU1Njvve975mTJ0+aP/qjPzLBYNBEIpESltz9hoaGzPvf/37z4Q9/2Gzbti1xnPouvAsXLpjbbrvNPPTQQ+Zf//VfzdjYmPnRj35k/uM//iNxDvVeWH/xF39hFi1aZH74wx+asbEx893vftf8zu/8jnn++ecT51Dn+fmHf/gH88wzz5jvfe97RpL5/ve/n/R6JvX76KOPmltuucUMDAyY4eFh83u/93vmd3/3d817772XV9lcHz5Wr15tHn300aRjy5cvN08//XSJSuRtExMTRpIZHBw0xhgTi8VMIBAwzz33XOKcqakp4/f7zTe/+c1SFdP1JicnzbJly8zAwIBZt25dInxQ38Wxfft2s3bt2rSvU++Ft2nTJvOFL3wh6dj9999vtmzZYoyhzgttZvjIpH5/9atfmfnz55sDBw4kzjl37pypqKgwBw8ezKs8ru52uXr1qo4fP64NGzYkHd+wYYOOHDlSolJ5WzgcliTV1dVJksbGxhQKhZI+g6qqKq1bt47PIA+PP/64Nm3apI9//ONJx6nv4nj11Vd155136jOf+Yzq6+u1YsUK7du3L/E69V54a9eu1Y9//GO9/fbbkqSf/exnOnz4sD75yU9Kos6LLZP6PX78uH7zm98kndPY2KiWlpa8PwPrNpbLxv/8z/8oGo2qoaEh6XhDQ4NCoVCJSuVdxhg9+eSTWrt2rVpaWiQpUc+pPoMzZ844XkYvOHDggIaHh3Xs2LFZr1HfxfHLX/5Svb29evLJJ/Wnf/qnGhoa0he/+EVVVVXp85//PPVeBNu3b1c4HNby5ctVWVmpaDSqZ599Vps3b5bEtV5smdRvKBTSDTfcoJtuumnWOfk+Y10dPuJ8Pl/Sn40xs44hf1u3btXPf/5zHT58eNZrfAaFcfbsWW3btk2vvfaaqqur055HfRdWLBbTnXfeqe7ubknSihUrdOrUKfX29urzn/984jzqvXC+853vqK+vT/v379cHP/hBnThxQh0dHWpsbNSDDz6YOI86L65c6rcQn4Gru11uvvlmVVZWzkpgExMTs9Ic8vPEE0/o1Vdf1euvv67FixcnjgcCAUniMyiQ48ePa2JiQqtWrdK8efM0b948DQ4O6m/+5m80b968RJ1S34UVDAbV3NycdOwDH/iA3nnnHUlc58Xw5S9/WU8//bQ++9nP6kMf+pD++I//WF/60pfU09MjiTovtkzqNxAI6OrVq7p48WLac3Ll6vBxww03aNWqVRoYGEg6PjAwoDVr1pSoVN5ijNHWrVv18ssv69ChQ2pqakp6vampSYFAIOkzuHr1qgYHB/kMcvCxj31MJ0+e1IkTJxI/d955pz73uc/pxIkTWrp0KfVdBPfcc8+sKeRvv/22brvtNklc58Vw6dIlVVQkP4IqKysTU22p8+LKpH5XrVql+fPnJ50zPj6ukZGR/D+DvIarWiA+1fbv/u7vzOjoqOno6DA33nij+c///M9SF80T2tvbjd/vN//8z/9sxsfHEz+XLl1KnPPcc88Zv99vXn75ZXPy5EmzefNmpsMV0LWzXYyhvothaGjIzJs3zzz77LPmF7/4hfn7v/97s3DhQtPX15c4h3ovrAcffNDccsstiam2L7/8srn55pvNU089lTiHOs/P5OSkefPNN82bb75pJJk9e/aYN998M7EURSb1++ijj5rFixebH/3oR2Z4eNjce++9TLWN+/rXv25uu+02c8MNN5iVK1cmpoEif5JS/rz44ouJc2KxmOns7DSBQMBUVVWZj370o+bkyZOlK7THzAwf1Hdx9Pf3m5aWFlNVVWWWL19uXnjhhaTXqffCikQiZtu2bebWW2811dXVZunSpeaZZ54xV65cSZxDnefn9ddfT3n/fvDBB40xmdXv5cuXzdatW01dXZ1ZsGCB+dSnPmXeeeedvMvmM8aY/NpOAAAAMufqMR8AAMB9CB8AAMBRhA8AAOAowgcAAHAU4QMAADiK8AEAABxF+AAAAI4ifAAAAEcRPgAAgKMIHwAAwFGEDwAA4CjCBwAAcNT/B+UsDRjhQUiKAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "np.random.seed(666)\n",
    "X = np.random.randint(0,100,(50,2)) # 生成一个50×2的矩阵，每一个元素在0~100随机生成整数\n",
    "plt.scatter(X[:,0], X[:,1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "31f3f672-04df-4558-9d01-68276c8a54b3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAALl1JREFUeJzt3X9sFPed//HX2gZvkuLtAcVrfoQzXNLiWk3ORiaGoKi54EAiN5x6Ck0uIcn1qjNtjwCXXKCc4hpFssg1ubv8wMnxI1EFISi/g+RzsZQLgeAcwtinUiOlZ9yYkN1aNt+unaQ2YXe+f1jrsngX76x3d2Z2ng9p//B4dvfj8ezMez/vz+f98RiGYQgAAMAieVY3AAAAuBvBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsFSB1Q1IRiQS0WeffaZp06bJ4/FY3RwAAJAEwzA0NDSk2bNnKy8vcf+HI4KRzz77TPPmzbO6GQAAIAVnz57V3LlzE/7eEcHItGnTJI3+MUVFRRa3BgAAJGNwcFDz5s0bu48n4ohgJJqaKSoqIhgBAMBhJhpiwQBWAABgKYIRAABgKYIRAABgKYIRAABgKYIRAABgKYIRAABgKYIRAABgKYIRAABgKUcUPQMAOwhHDB3vOa++oWHNmuZVVel05eexXhYwWaZ7Rj744APV1tZq9uzZ8ng8evvttyd8zuHDh1VZWSmv16sFCxbohRdeSKWtAGCZllMB3bz9Pd2z8yM9/Gqn7tn5kW7e/p5aTgWsbhrgeKaDkS+++EI33HCDnnvuuaT27+np0R133KHly5ero6NDP/vZz7R+/Xq98cYbphsLAFZoORXQur0nFQgNx2wPhoa1bu9JAhJgkkynaVatWqVVq1Ylvf8LL7yga6+9Vv/+7/8uSVq0aJFOnDihX/ziF/r+979v9u0BIKvCEUMNB7tkxPmdIckjqeFgl1aU+UnZACnK+ADWtrY21dTUxGy7/fbbdeLECX311VdxnzMyMqLBwcGYBwBY4XjP+XE9IpcyJAVCwzrecz57jQJyTMaDkWAwqOLi4phtxcXFunjxovr7++M+p7GxUT6fb+wxb968TDcTAOLqG0ociKSyH4DxsjK19/Klgw3DiLs9asuWLQqFQmOPs2fPZryNABDPrGnetO4HYLyMT+31+/0KBoMx2/r6+lRQUKAZM2bEfU5hYaEKCwsz3TQAmFBV6XSV+LwKhobjjhvxSPL7Rqf5AkhNxntGqqur1draGrPt0KFDWrx4saZMmZLpt4eFwhFDbd0DeqfznNq6BxSOxLuUA/aWn+dRfW2ZpNHA41LRn+tryxi8CkyC6Z6Rzz//XP/3f/839nNPT486Ozs1ffp0XXvttdqyZYvOnTunX/7yl5Kkuro6Pffcc9q0aZN+9KMfqa2tTbt379b+/fvT91fAdlpOBdRwsCtm4F+Jz6v62jKtLC+xsGWAeSvLS9R0X8W4c9rPOQ2khceIDuBI0vvvv6/vfve747Y/8MADevnll/Xggw/qd7/7nd5///2x3x0+fFgbN27Ub37zG82ePVuPPfaY6urqkn7PwcFB+Xw+hUIhFRUVmWkuLBCtyXD5iRX93th0XwUXbzgSFVgBc5K9f5sORqxAMOIc4Yihm7e/l3AqZDS/fvSxW7mIA0COS/b+zUJ5SCtqMgAAzCIYQVpRkwEAYBbBCNKKmgwAALMyXmfErhiIlhnUZIDbcC0BJs+VwQjTTjMnWpNh3d6T8kgxAQk1GZBruJYA6eG6NA1LgWdetCaD3xebivH7vEzrRc7gWgKkj6um9jLtNLvovkau4loCJCfZ+7er0jRmpp1WL4y/bg6Sl5/n4TgiJ3EtAdLLVWkapp0CSAeuJUB6uSoYYdopgHTgWgKkl6uCkei000QZXI9GR8Iz7RTAlXAtAdLLVcEIS4EDSAeuJUB6uSoYkZh2CiA9uJYA6eOqqb2XYtopgHTgWgIkxtTeCTDtFEA6cC0BJs+1wQjfZgAAsAdXBiOsJwEAgH24bgAr60kAAGAvrgpGwhFDDQe74i5tH93WcLBL4Yjtx/QCAJAzXBWMmFlPAgAAZIerghHWkwAAwH5cFYywngQAAPbjqtk00fUkgqHhuONGPBqtnsh6EkgF08UBIDWuCkai60ms23tSHikmIGE9CUwG08UBIHWuStNIrCeB9GO6OABMjqt6RqJWlpdoRZmfLnVM2kTTxT0anS6+oszP+QUACbgyGJFYTwLpYWa6OOcbAMTnujQNkE5MFweAySMYASaB6eIAMHkEI8AkRKeLJxoN4tHorBqmiwNAYgQjwCREp4tLGheQMF0888IRQ23dA3qn85zaugdYVwpwKNcOYAXSJTpd/PI6I37qjGQUtV2A3OExDMP2XyUGBwfl8/kUCoVUVFRkdXOAuKjAmj3R2i6XX7yiR5uaQYA9JHv/pmcESBOmi2cHtV2QS6z+EmP1+0cRjABwFGq7IFdYnWq0+v0vxQBWAI5CbRfkAquXkbD6/S9HMALAUajtAqebKNUojaYaMzU7zOr3j4dgBICjUNsFTmcm1ZiL7x8PwQgAR6G2C5zO6lSj1e8fD8EIAMeJ1nbx+2JTMX6fl2m9sD2rU41Wv388zKYB4Egry0u0osxvi2mJgBnRVGMwNBx33IZHo4F1plKNVr9/PPSMAHCsaG2Xu26co+qFMwhE4AhWpxqtfv94CEYAAMgyq1ONVr//5SgHDwCARayugJrp96ccPAAANmf1MhJWv38UaRoAAGApghEAAGApghEAAGApxowgZ1g9EAwAkBqCEeQEOy2FDQAwhzQNHM9uS2EDAMwhGIGj2XEpbACAOQQjcDQ7LoUNADCHYASOZselsAEA5hCMwNHsuBQ2AMAcghE4WnQp7EQTeD0anVWTzaWwAQDmEIzA0ey4FDYAwJyUgpEdO3aotLRUXq9XlZWVOnLkyBX337dvn2644QZdffXVKikp0UMPPaSBgYGUGgxczm5LYQMAzPEYhmFqzuOBAwd0//33a8eOHVq2bJlefPFF7dq1S11dXbr22mvH7X/06FHdcsst+rd/+zfV1tbq3Llzqqur03XXXae33norqfdMdgliuBsVWAHAXpK9f5sORpYsWaKKigo1NTWNbVu0aJFWr16txsbGcfv/4he/UFNTk7q7u8e2Pfvss3ryySd19uzZpN6TYAQAkAhfROwr2fu3qXLwFy5cUHt7uzZv3hyzvaamRseOHYv7nKVLl2rr1q1qbm7WqlWr1NfXp9dff1133nlnwvcZGRnRyMhIzB8DAMDlWAoiN5gaM9Lf369wOKzi4uKY7cXFxQoGg3Gfs3TpUu3bt09r1qzR1KlT5ff79fWvf13PPvtswvdpbGyUz+cbe8ybN89MMwEALsBSELkjpQGsHk9s95dhGOO2RXV1dWn9+vV6/PHH1d7erpaWFvX09Kiuri7h62/ZskWhUGjskWw6BwDgDiwFkVtMpWlmzpyp/Pz8cb0gfX1943pLohobG7Vs2TI9+uijkqTvfOc7uuaaa7R8+XI98cQTKikZ341WWFiowsJCM00DALiImaUgqhfOyF7DkBJTPSNTp05VZWWlWltbY7a3trZq6dKlcZ/z5ZdfKi8v9m3y8/MljfaoAABgFktB5BbTaZpNmzZp165d2rNnj06fPq2NGzeqt7d3LO2yZcsWrV27dmz/2tpavfnmm2pqatKZM2f04Ycfav369aqqqtLs2bPT95cAAFyDpSByi6k0jSStWbNGAwMD2rZtmwKBgMrLy9Xc3Kz58+dLkgKBgHp7e8f2f/DBBzU0NKTnnntO//RP/6Svf/3ruvXWW7V9+/b0/RUAAFeJLgURDA3HHTfi0WjhQ5aCcAbTdUasQJ0RAMDlorNpJMUEJNHpFFRgtl6y92/WpgEAOBJLQeQO02kaAADsYmV5iVaU+anA6nAEIwAAR8vP8zB91+EIRgAgx7F2C+yOYAQAchhrt8AJGMAKADmKtVvgFAQjAJCDWLsFTkIwAgA5yMzaLYDVCEYAIAexdguchGAEAHIQa7fASZhNY3NMyQOQCtZugZMQjNgYU/IApCo/z6P62jKt23tSHsVfu6W+towvN7AF0jQ2xZQ8AJPF2i1wCnpGbGiiKXkejU7JW1Hm51sNgCti7RY4AcGIDZmZksd6DAAmwtotsDvSNDbElDwAgJsQjNgQU/IAAG5CMGJD0Sl5iTK6Ho3OqmFKHgAgFxCM2FB0Sp6kcQGJFVPywhFDbd0DeqfznNq6B1jLwgL8DwB74zM6OQxgtanolLzL64z4s1xnhFon1uN/ANgbn9HJ8xiGYfvwbXBwUD6fT6FQSEVFRVY3J6usrMAarXVy+QkSfXfqFGQe/wPA3viMXlmy92/SNDYXnZJ3141zVL1wRlZTMyw/bi3+B4C98RlNH4IRxMXy49bjfwDYG5/R9CEYQVzUOrEe/wPA3viMpg/BCOKi1on1+B8A9sZnNH0IRhAXtU6sx/8AsDc+o+lDMIK47FbrxI34HwD2xmc0fQhGkBDLj1uP/wFgb3xG04M6I5iQlbVOMIr/AWBvfEbjS/b+TQVWTIjlx63H/wCwNz6jk0OaBgAAWIpgBAAAWIpgBAAAWIpgBAAAWIpgBAAAWIpgBAAAWIqpvUAKMl1TgJoFAKLccD0gGAFMajkVUMPBrpilw0t8XtXXlqWl2mKmXx+Ac7jlekCaBjCh5VRA6/aejLkwSFIwNKx1e0+q5VTA1q8PwDncdD0gGAGSFI4YajjYpXjrJ0S3NRzsUjiS2goLmX59AM7htusBwQhsLxwx1NY9oHc6z6mte8CyD9/xnvPjvqFcypAUCA3reM95W74+AOdIx/XALtfOZDBmBLZmp3xp31DiC0Mq+2X79QE4x2SvB3a6diaDnhHYlt3ypbOmeSfeycR+2X59AM4xmeuB3a6dySAYgS3ZMV9aVTpdJT6vEk2o82j0m0dV6XRbvj4A50j1emDHa2cyCEZgS9kcP5FsXjU/z6P62jJJGneBiP5cX1uW8vz/TL8+AOdI9Xrg1LFnBCOwpWyNn2g5FdDN29/TPTs/0sOvduqenR/p5u3vJezGXFleoqb7KuT3xXaN+n1eNd1XMelcbKZfH4BzpHI9cOrYMwawwpayMX4imle9vB8kmldN9GFfWV6iFWX+jFVEzPTrA3AOs9cDp449IxiBLUXzpcHQcNzcp0ej3w5SHT8xUV7Vo9G86ooyf9wPfX6eR9ULZ6T03snI9OsDcA4z14NMXzszhTQNbCnT4yecmlcFgCtx6tgzghHYVibHTzg1rwoAE3Hi2DPSNLC1TI2fcGpeFQCS4bSxZwQjsL1MjJ9wal4VAJLlpLFnpGngSk7NqwJALiIYgWs5Ma8KALmINA1czWl5VQDIRQQjcD0n5VUBIBellKbZsWOHSktL5fV6VVlZqSNHjlxx/5GREW3dulXz589XYWGhFi5cqD179qTUYAAAkFtM94wcOHBAGzZs0I4dO7Rs2TK9+OKLWrVqlbq6unTttdfGfc7dd9+t3//+99q9e7f+4i/+Qn19fbp48eKkGw8AAJzPYxiGqXWElyxZooqKCjU1NY1tW7RokVavXq3GxsZx+7e0tOgHP/iBzpw5o+nTU5smOTg4KJ/Pp1AopKKiopReAwAAZFey929TaZoLFy6ovb1dNTU1Mdtramp07NixuM959913tXjxYj355JOaM2eOrr/+ej3yyCP64x//mPB9RkZGNDg4GPMAAKuFI4baugf0Tuc5tXUPKBwx9V0OQAKm0jT9/f0Kh8MqLi6O2V5cXKxgMBj3OWfOnNHRo0fl9Xr11ltvqb+/Xz/+8Y91/vz5hONGGhsb1dDQYKZpAJBRLacCajjYFbOmUYnPq/raMqaBA5OU0gBWjyd22qNhGOO2RUUiEXk8Hu3bt09VVVW644479PTTT+vll19O2DuyZcsWhUKhscfZs2dTaSYApEXLqYDW7T05bnHFYGhY6/aeVMupgEUtA3KDqWBk5syZys/PH9cL0tfXN663JKqkpERz5syRz+cb27Zo0SIZhqFPP/007nMKCwtVVFQU8wAAK4QjhhoOdsVdNiC6reFgFymbDCE15g6m0jRTp05VZWWlWltb9dd//ddj21tbW3XXXXfFfc6yZcv02muv6fPPP9fXvvY1SdLHH3+svLw8zZ07dxJNB4DMO95zflyPyKUMSYHQsI73nKdeTZqRGnMP02maTZs2adeuXdqzZ49Onz6tjRs3qre3V3V1dZJGUyxr164d2//ee+/VjBkz9NBDD6mrq0sffPCBHn30Uf3d3/2drrrqqvT9JQCQAX1DiQORVPZDckiNuYvpOiNr1qzRwMCAtm3bpkAgoPLycjU3N2v+/PmSpEAgoN7e3rH9v/a1r6m1tVX/+I//qMWLF2vGjBm6++679cQTT6TvrwCADJk1zTvxTib2w8QmSo15NJoaW1HmZ+mGHGG6zogVqDMCwCrhiKGbt7+nYGg47s3Ro9HFFY8+dis3xjRp6x7QPTs/mnC//T+6idSYzWWkzggAuE1+nkf1tWWSRgOPS0V/rq8tIxBJI1Jj7kMwAgATWFleoqb7KuT3xaZi/D6vmu6rYDBlmpEacx9W7QWAJKwsL9GKMr+O95xX39CwZk3zqqp0Oj0iGVBVOl0lPu+EqbGq0tSWGIH9EIwAQJLy8zyMUciCaGps3d6T8kgxAQmpsdxEmgYAYDukxtyFnhEAgC2RGnMPghEAgG2RGnMH0jQAAMBSBCMAAMBSBCMAAMBSBCMAAMBSBCMAAMBSBCMAAMBSBCMAAMBS1BlBWoQjBoWJAAApIRjBpLWcCqjhYJcCoT8t513i86q+toySzQCACZGmwaS0nApo3d6TMYGIJAVDw1q396RaTgUsahkAwCkIRpCycMRQw8GuuEt8R7c1HOxSOBJvDwAARpGmsZDTx1kc7zk/rkfkUoakQGhYx3vOs7YEAKTI6feKZBCMWCQXxln0DSUORFLZDwAQKxfuFckgTWOBXBlnMWuaN637AQD+JFfuFckgGMmyXBpnUVU6XSU+rxJ1Fno0GsFXlU7PZrMAwPFy6V6RDIKRLDMzzsLu8vM8qq8tk6RxAUn05/raspzLbQJApuXSvSIZBCNZlmvjLFaWl6jpvgr5fbGpGL/Pq6b7KnIqpwkA2ZJr94qJMIA1y3JxnMXK8hKtKPPn/GhvAMiWXLxXXAnBSJZFx1kEQ8Nxc4EejfYqOG2cRX6eh+m7AJAmuXqvSIQ0TZYxzgIAMBG33SsIRizAOAsAwETcdK/wGIZh+3lBg4OD8vl8CoVCKioqsro5aeOGqnoAgMlx8r0i2fs3Y0YsxDgLAMBE3HCvIE0DAAAsRTACAAAsRTACAAAsxZgRwKGcPKgNAC5FMAI4kFuWFQfgDqRpAIdx07LiANyBnhEXo5vfeSZaVtyj0WXFV5T5+V8COcIN12qCEZeim9+ZzCwrnut1CQA3cMu1mjSNC9HN71xuW1YccDM3XasJRlxmom5+abSbPxyx/SoBruS2ZcUBt3Lbtdq1wUg4Yqite0DvdJ5TW/dAzvxDJ2Kmmx/2E11WPFG22KPRLtzosuJuPc9hP5yL5rjtWu3KMSNuycHFQze/s0WXFV+396Q8Usy3psuXFXfzeQ574Vw0z23Xatf1jLgpBxcP3fzOl8yy4m4/z2EfnIupcdu12lU9I0yL/FM3fzA0HPc4eDR6U4t288OeVpaXaEWZP+50P85z2AXnYurcdq12Vc+I23Jw8US7+SWNG3dweTc/7C26rPhdN85R9cIZY/8zznPYBedi6tx2rXZVMOK2HFwiyXTzw7k4z2EXnIuT46ZrtavSNG7LwV3Jlbr54Wyc57ALzsXJc8u12lXBiNtycBOJdvMjt3Cewy44F9PDDddqV6Vp3JaDgztxnsMuOBeRLFcFI5K7cnBwL85z2AXnIpLhMQzD9mXwBgcH5fP5FAqFVFRUlJbXdMMqiADnOeyCc9Gdkr1/u2rMyKXckIMDOM9hF5yLuBLXpWkAAIC9EIwAAABLEYwAAABLEYwAAABLpRSM7NixQ6WlpfJ6vaqsrNSRI0eSet6HH36ogoIC3Xjjjam8LQAAyEGmg5EDBw5ow4YN2rp1qzo6OrR8+XKtWrVKvb29V3xeKBTS2rVr9Vd/9VcpNxYAAOQe03VGlixZooqKCjU1NY1tW7RokVavXq3GxsaEz/vBD36g6667Tvn5+Xr77bfV2dmZ9Htmos4IAADIrGTv36Z6Ri5cuKD29nbV1NTEbK+pqdGxY8cSPu+ll15Sd3e36uvrk3qfkZERDQ4OxjwAAEBuMhWM9Pf3KxwOq7i4OGZ7cXGxgsFg3Of89re/1ebNm7Vv3z4VFCRXY62xsVE+n2/sMW/ePDPNBAAADpLSAFaPJ7aEr2EY47ZJUjgc1r333quGhgZdf/31Sb/+li1bFAqFxh5nz55NpZkAAMABTJWDnzlzpvLz88f1gvT19Y3rLZGkoaEhnThxQh0dHfrpT38qSYpEIjIMQwUFBTp06JBuvfXWcc8rLCxUYWGhmaYBAACHMtUzMnXqVFVWVqq1tTVme2trq5YuXTpu/6KiIv36179WZ2fn2KOurk7f/OY31dnZqSVLlkyu9QAAwPFML5S3adMm3X///Vq8eLGqq6v1n//5n+rt7VVdXZ2k0RTLuXPn9Mtf/lJ5eXkqLy+Pef6sWbPk9XrHbQcAAO5kOhhZs2aNBgYGtG3bNgUCAZWXl6u5uVnz58+XJAUCgQlrjgAAAESZrjNiBeqMAADMCEcMHe85r76hYc2a5lVV6XTl542faIHMSvb+bbpnBAAAO2s5FVDDwS4FQsNj20p8XtXXlmlleYmFLUMiLJQHAMgZLacCWrf3ZEwgIknB0LDW7T2pllMBi1qGKyEYAQDkhHDEUMPBLsUbexDd1nCwS+GI7UcnuA7BCAAgJxzvOT+uR+RShqRAaFjHe85nr1FICsEIACAn9A0lDkRS2Q/ZQzACAMgJs6Z507ofsodgBACQE6pKp6vE51WiCbwejc6qqSqdns1mIQlM7UVaOHlOv5PbbjccSySSjXMjP8+j+toyrdt7Uh4pZiBr9J3qa8s4J22IYAST5uQ5/U5uu91wLJFINs+NleUlarqvYtz7+TkXbY0KrJiU6Jz+y0+i6PeOpvsqbPvhd3Lb7YZjiUSsOjfopbOHZO/fjBlBypw8p9/JbbcbjiUSsfLcyM/zqHrhDN114xxVL5xBIGJzBCNImZPn9Du57XbDsUQinBtIFsEIUubkOf1ObrvdcCyRCOcGkkUwgpQ5eU6/k9tuNxxLJMK5gWQRjCBlTp7T7+S22w3HEolwbiBZBCNIWXROv6RxFxu7z+l3ctvthmOJRDg3kCyCEUxKdE6/3xfbzer3eW0/ndPJbbcbjiUS4dxAMqgzgrRw8px+J7fdbjiWSIRzw52SvX8TjAAAgIyg6BkAAHAEghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGCpAqsbAGC8cMTQ8Z7z6hsa1qxpXlWVTld+nsfqZgFARhCMADbTciqghoNdCoSGx7aV+Lyqry3TyvISC1sGAJlBmgawkZZTAa3bezImEJGkYGhY6/aeVMupgEUtA4DMIRgBbCIcMdRwsEtGnN9FtzUc7FI4Em8PAHAughHAJo73nB/XI3IpQ1IgNKzjPeez1ygAyAKCEcAm+oYSByKp7AcATkEwAtjErGnetO4HAE5BMALYRFXpdJX4vEo0gdej0Vk1VaXTs9ksAMg4ghGTwhFDbd0DeqfznNq6BxhMiLTJz/OovrZMksYFJNGf62vLqDcCIOdQZ8QE6j8g01aWl6jpvopx55mf8wxADvMYhmH7r/aDg4Py+XwKhUIqKiqypA3R+g+XH6zod9Sm+yq4USBtqMAKIBcke/+mZyQJE9V/8Gi0/sOKMj83DKRFfp5H1QtnWN0MAMgKxowkwcr6D4xRAQDkOnpGkmBV/QfGqAAA3ICekSRYUf+BNUoAAG5BMJKEbNd/YI0SILeQbgWuLKVgZMeOHSotLZXX61VlZaWOHDmScN8333xTK1as0De+8Q0VFRWpurpav/rVr1JusBWyXf+BNUqA3NFyKqCbt7+ne3Z+pIdf7dQ9Oz/Szdvfo3cTuITpYOTAgQPasGGDtm7dqo6ODi1fvlyrVq1Sb29v3P0/+OADrVixQs3NzWpvb9d3v/td1dbWqqOjY9KNz6Zo/Qe/LzYV4/d50z6tlzVKgNxAuhVIjuk6I0uWLFFFRYWamprGti1atEirV69WY2NjUq/x7W9/W2vWrNHjjz+e1P52qDMSlY36D23dA7pn50cT7rf/Rzcx/ROwqXDE0M3b30vYy+nR6JeZo4/dSkkA5KyM1Bm5cOGC2tvbtXnz5pjtNTU1OnbsWFKvEYlENDQ0pOnTE4+vGBkZ0cjIyNjPg4ODZpqZUdmo/xAdoxIMDccdNxK9iLFGCWBfZtKtfKmA25lK0/T39yscDqu4uDhme3FxsYLBYFKv8dRTT+mLL77Q3XffnXCfxsZG+Xy+sce8efPMNNPxWKMEcD7SrUDyUhrA6vHE3gQNwxi3LZ79+/fr5z//uQ4cOKBZs2Yl3G/Lli0KhUJjj7Nnz6bSTEfL5hgVAOlnRUkAwKlMpWlmzpyp/Pz8cb0gfX1943pLLnfgwAH98Ic/1GuvvabbbrvtivsWFhaqsLDQTNNy0sryEq0o87NGCeBApFuB5JnqGZk6daoqKyvV2toas721tVVLly5N+Lz9+/frwQcf1CuvvKI777wztZa6VHSMyl03zlH1whkEIoBDkG4Fkmc6TbNp0ybt2rVLe/bs0enTp7Vx40b19vaqrq5O0miKZe3atWP779+/X2vXrtVTTz2lm266ScFgUMFgUKFQKH1/BQDYEOlWIDmm16ZZs2aNBgYGtG3bNgUCAZWXl6u5uVnz58+XJAUCgZiaIy+++KIuXryon/zkJ/rJT34ytv2BBx7Qyy+/PPm/AABsjHQrMDHTdUasYKc6IwBgd9mohwQkIyN1RgAA9sZq33AiFsoDgBxB+Xk4FcEIAOQAVvuGkxGMAEhZOGKorXtA73SeU1v3ADc6C7HaN5yMMSMAUsLYBHuh/DycjJ4RAKYxNsF+KD8PJyMYAWAKYxPsKVp+PtEEXo9Ge64oPw87IhgBYApjE+yJ8vNwMoIRAKYwNsG+KD8Pp2IAKwBTGJtgb5SfhxMRjAAwJTo2IRgajjtuxKPRb+KMTbBOdLVvwClI0wAwhbEJANKNYASAaYxNAJBOpGkApISxCQDShZ4RAABgKXpGAKSEcvAA0oWeEQCmUQ4eQDoRjAAwhXLwANKNYASAKZSDB5BuBCMATKEcPIB0IxgBYArl4AGkG8EIAFNYqh5AuhGMIGeEI4baugf0Tuc5tXUP6MLFSMzPDKhMD8rBA0g36owgJ8SreZHnkS6NP6iBkT7RcvCXH3M/xxhACjyGYdj+6+Lg4KB8Pp9CoZCKioqsbg5sJlrzYqITOfo9nbVT0iccMSgHDyChZO/f9IzA0a5U8+JyhkYDkoaDXVpR5uemmQYsVQ8gHRgzAkebqObF5aiBAdjX5eO+GOflHvSMwNFSrWVBDQzAXljryN3oGYGjpVrLghoYgH2w1hEIRiZAt6G9TVTz4nLUwADshbWOIJGmuSK6De0vWvNi3d6T8khXHMhKDQzAfsysdcRg6dxFz0gCdBs6R7Tmhd8Xm3q5PN7w+7xM6wVshrWOINEzEtdE3YZMD7WfleUlWlHmj6l5UTn/z9T+yf+jBgZgY6x1BIlgJC66DZ0pXs0L/j+AvUXHfQVDw3G/AHo02qvJOK/cRpomDroNASA7WOsIEsFIXHQbAkD2JBr3xTgv9yBNEwfdhgCQXfHGfTHOyz0IRuK40nRRug0BIDNY68i9SNMkQLchAADZQc/IFdBtCABA5hGMTIBuQwAAMos0DQAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsBTBCAAAsFSB1Q0AUhGOGDrec159Q8OaNc2rqtLpys/zWN0sAEAKUuoZ2bFjh0pLS+X1elVZWakjR45ccf/Dhw+rsrJSXq9XCxYs0AsvvJBSYwFJajkV0M3b39M9Oz/Sw6926p6dH+nm7e+p5VTA6qYBAFJgOhg5cOCANmzYoK1bt6qjo0PLly/XqlWr1NvbG3f/np4e3XHHHVq+fLk6Ojr0s5/9TOvXr9cbb7wx6cbDfVpOBbRu70kFQsMx24OhYa3be5KABAAcyGMYhmHmCUuWLFFFRYWamprGti1atEirV69WY2PjuP0fe+wxvfvuuzp9+vTYtrq6Ov3v//6v2traknrPwcFB+Xw+hUIhFRUVmWkuckg4Yujm7e+NC0SiPJL8Pq+OPnYrKRsAsIFk79+mekYuXLig9vZ21dTUxGyvqanRsWPH4j6nra1t3P633367Tpw4oa+++iruc0ZGRjQ4OBjzAI73nE8YiEiSISkQGtbxnvPZaxQAYNJMBSP9/f0Kh8MqLi6O2V5cXKxgMBj3OcFgMO7+Fy9eVH9/f9znNDY2yufzjT3mzZtnppnIUX1DiQORVPYDANhDSgNYPZ7YLnDDMMZtm2j/eNujtmzZolAoNPY4e/ZsKs1Ejpk1zZvW/QAA9mBqau/MmTOVn58/rhekr69vXO9HlN/vj7t/QUGBZsyYEfc5hYWFKiwsNNM0uEBV6XSV+LwKhoYVb6BTdMxIVen0bDcNADAJpnpGpk6dqsrKSrW2tsZsb21t1dKlS+M+p7q6etz+hw4d0uLFizVlyhSTzYWb5ed5VF9bJmk08LhU9Of62jIGrwKAw5hO02zatEm7du3Snj17dPr0aW3cuFG9vb2qq6uTNJpiWbt27dj+dXV1+uSTT7Rp0yadPn1ae/bs0e7du/XII4+k76+Aa6wsL1HTfRXy+2JTMX6fV033VWhleYlFLQMApMp0BdY1a9ZoYGBA27ZtUyAQUHl5uZqbmzV//nxJUiAQiKk5UlpaqubmZm3cuFHPP/+8Zs+erWeeeUbf//730/dXwFVWlpdoRZmfCqwAkCNM1xmxAnVGAABwnozUGQEAAEg3ghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGApghEAAGAp0+XgrRAtEjs4OGhxSwAAQLKi9+2Jir07IhgZGhqSJM2bN8/ilgAAALOGhobk8/kS/t4Ra9NEIhF99tlnmjZtmjye9C2GNjg4qHnz5uns2bOseZNhHOvs4DhnB8c5OzjO2ZHJ42wYhoaGhjR79mzl5SUeGeKInpG8vDzNnTs3Y69fVFTEiZ4lHOvs4DhnB8c5OzjO2ZGp43ylHpEoBrACAABLEYwAAABLuToYKSwsVH19vQoLC61uSs7jWGcHxzk7OM7ZwXHODjscZ0cMYAUAALnL1T0jAADAegQjAADAUgQjAADAUgQjAADAUjkfjOzYsUOlpaXyer2qrKzUkSNHrrj/4cOHVVlZKa/XqwULFuiFF17IUkudzcxxfvPNN7VixQp94xvfUFFRkaqrq/WrX/0qi611NrPndNSHH36ogoIC3XjjjZltYI4we5xHRka0detWzZ8/X4WFhVq4cKH27NmTpdY6l9njvG/fPt1www26+uqrVVJSooceekgDAwNZaq0zffDBB6qtrdXs2bPl8Xj09ttvT/icrN8LjRz26quvGlOmTDF27txpdHV1GQ8//LBxzTXXGJ988knc/c+cOWNcffXVxsMPP2x0dXUZO3fuNKZMmWK8/vrrWW65s5g9zg8//LCxfft24/jx48bHH39sbNmyxZgyZYpx8uTJLLfcecwe66g//OEPxoIFC4yamhrjhhtuyE5jHSyV4/y9733PWLJkidHa2mr09PQY//M//2N8+OGHWWy185g9zkeOHDHy8vKM//iP/zDOnDljHDlyxPj2t79trF69Osstd5bm5mZj69atxhtvvGFIMt56660r7m/FvTCng5Gqqiqjrq4uZtu3vvUtY/PmzXH3/+d//mfjW9/6Vsy2f/iHfzBuuummjLUxF5g9zvGUlZUZDQ0N6W5azkn1WK9Zs8b4l3/5F6O+vp5gJAlmj/N//dd/GT6fzxgYGMhG83KG2eP8r//6r8aCBQtitj3zzDPG3LlzM9bGXJNMMGLFvTBn0zQXLlxQe3u7ampqYrbX1NTo2LFjcZ/T1tY2bv/bb79dJ06c0FdffZWxtjpZKsf5cpFIRENDQ5o+fXommpgzUj3WL730krq7u1VfX5/pJuaEVI7zu+++q8WLF+vJJ5/UnDlzdP311+uRRx7RH//4x2w02ZFSOc5Lly7Vp59+qubmZhmGod///vd6/fXXdeedd2ajya5hxb3QEQvlpaK/v1/hcFjFxcUx24uLixUMBuM+JxgMxt3/4sWL6u/vV0lJScba61SpHOfLPfXUU/riiy909913Z6KJOSOVY/3b3/5Wmzdv1pEjR1RQkLMf97RK5TifOXNGR48eldfr1VtvvaX+/n79+Mc/1vnz5xk3kkAqx3np0qXat2+f1qxZo+HhYV28eFHf+9739Oyzz2ajya5hxb0wZ3tGojweT8zPhmGM2zbR/vG2I5bZ4xy1f/9+/fznP9eBAwc0a9asTDUvpyR7rMPhsO699141NDTo+uuvz1bzcoaZczoSicjj8Wjfvn2qqqrSHXfcoaefflovv/wyvSMTMHOcu7q6tH79ej3++ONqb29XS0uLenp6VFdXl42mukq274U5+1Vp5syZys/PHxdh9/X1jYv4ovx+f9z9CwoKNGPGjIy11clSOc5RBw4c0A9/+EO99tpruu222zLZzJxg9lgPDQ3pxIkT6ujo0E9/+lNJozdNwzBUUFCgQ4cO6dZbb81K250klXO6pKREc+bMiVkqfdGiRTIMQ59++qmuu+66jLbZiVI5zo2NjVq2bJkeffRRSdJ3vvMdXXPNNVq+fLmeeOIJeq/TxIp7Yc72jEydOlWVlZVqbW2N2d7a2qqlS5fGfU51dfW4/Q8dOqTFixdrypQpGWurk6VynKXRHpEHH3xQr7zyCvneJJk91kVFRfr1r3+tzs7OsUddXZ2++c1vqrOzU0uWLMlW0x0llXN62bJl+uyzz/T555+Pbfv444+Vl5enuXPnZrS9TpXKcf7yyy+Vlxd728rPz5f0p2/umDxL7oUZGxprA9FpY7t37za6urqMDRs2GNdcc43xu9/9zjAMw9i8ebNx//33j+0fnc60ceNGo6ury9i9ezdTe5Ng9ji/8sorRkFBgfH8888bgUBg7PGHP/zBqj/BMcwe68sxmyY5Zo/z0NCQMXfuXONv/uZvjN/85jfG4cOHjeuuu874+7//e6v+BEcwe5xfeuklo6CgwNixY4fR3d1tHD161Fi8eLFRVVVl1Z/gCENDQ0ZHR4fR0dFhSDKefvppo6OjY2wKtR3uhTkdjBiGYTz//PPG/PnzjalTpxoVFRXG4cOHx373wAMPGLfcckvM/u+//77xl3/5l8bUqVONP//zPzeampqy3GJnMnOcb7nlFkPSuMcDDzyQ/YY7kNlz+lIEI8kze5xPnz5t3HbbbcZVV11lzJ0719i0aZPx5ZdfZrnVzmP2OD/zzDNGWVmZcdVVVxklJSXG3/7t3xqffvppllvtLP/93/99xWuuHe6FHsOgbwsAAFgnZ8eMAAAAZyAYAQAAliIYAQAAliIYAQAAliIYAQAAliIYAQAAliIYAQAAliIYAQAAliIYAQAAliIYAQAAliIYAQAAliIYAQAAlvr/AnwFKsQoID8AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = np.array(X, dtype=float)\n",
    "X[:,0] = (X[:,0] - np.min(X[:,0])) / (np.max(X[:,0]) - np.min(X[:,0]))\n",
    "X[:,1] = (X[:,1] - np.min(X[:,1])) / (np.max(X[:,1]) - np.min(X[:,1]))\n",
    "plt.scatter(X[:,0], X[:,1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb6d039e-dd89-4e36-94c3-fb506a0c4814",
   "metadata": {},
   "source": [
    "**（2）均值方差归一化**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "3a38221a-ff23-4b90-8055-da1a09cebed9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAGdCAYAAAA44ojeAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAALOVJREFUeJzt3X1wVNd9//HPSsAKqLSNoGglg4nwqGPLchIDgZ8xDcQ2Cg1R4vE0dYxx7EnHw6ODTBseSlrAE0uGtsTTkCiFyThuVYKnEzs2GYeixKlsCi4EoQShjh0TxRCsHTVAd+UYSbF0fn+ou/ZaEmhXd3fPvff9mtEfunsR5x7tw0fne+45AWOMEQAAgEXyct0AAACADyKgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsMy7XDUjHwMCA3nrrLRUWFioQCOS6OQAAYBSMMeru7lZZWZny8q4+RuLKgPLWW29pxowZuW4GAABIw/nz5zV9+vSrnuPKgFJYWChp8AKLiopy3BoAADAasVhMM2bMSHyOX40rA0q8rFNUVERAAQDAZUYzPYNJsgAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1kk5oLz88suqqalRWVmZAoGAfvCDHyQ9bozR9u3bVVZWpokTJ2rx4sU6c+ZM0jm9vb165JFHNHXqVE2ePFmf/exn9Zvf/GZMFwIAALwj5YDyu9/9Th/96Ee1Z8+eYR/ftWuXdu/erT179ujEiRMKh8NasmSJuru7E+fU1tbqueee04EDB3TkyBG9/fbb+sxnPqP+/v70rwQAAHhGwBhj0v7HgYCee+453X333ZIGR0/KyspUW1urTZs2SRocLSkpKdHOnTu1cuVKRaNR/dEf/ZH+5V/+Rffee6+k95auf/HFF/WpT33qmv9vLBZTKBRSNBploTbAIv0DRsc7Lqmru0fTCgs0r7xY+XnslwVgUCqf346uJNvR0aFIJKLq6urEsWAwqEWLFuno0aNauXKlTp48qd///vdJ55SVlamqqkpHjx4dNqD09vaqt7c38X0sFnOy2QAccKitUzsOtqsz2pM4Vhoq0LaaSi2tKs1hywC4kaOTZCORiCSppKQk6XhJSUnisUgkogkTJuhDH/rQiOd8UH19vUKhUOKLjQIBuxxq69TqxpakcCJJkWiPVje26FBbZ45aBsCtMnIXzwfX2DfGXHPd/auds2XLFkWj0cTX+fPnHWsrgLHpHzDacbBdw9WK48d2HGxX/0Da1WQAPuRoQAmHw5I0ZCSkq6srMaoSDofV19eny5cvj3jOBwWDwcTGgGwQCNjleMelISMn72ckdUZ7dLzjUvYa5SP9A0bHzl7U860XdOzsRYIgPMPRgFJeXq5wOKympqbEsb6+PjU3N2vBggWSpDlz5mj8+PFJ53R2dqqtrS1xDgD36OoeOZykcx5G71BbpxbufEn37XtV6w+06r59r2rhzpcoqcETUp4k+/bbb+uNN95IfN/R0aHW1lYVFxfr+uuvV21trerq6lRRUaGKigrV1dVp0qRJWr58uSQpFArpL/7iL/SXf/mXmjJlioqLi/VXf/VXuuWWW3TXXXc5d2UAsmJaYYGj52F04vN+PjheEp/307BiNpOT4WopB5Sf/exn+uQnP5n4fsOGDZKkBx98UN/97ne1ceNGXblyRWvWrNHly5c1f/58HT58WIWFhYl/8/Wvf13jxo3Tn//5n+vKlSu688479d3vflf5+fkOXBKAbJpXXqzSUIEi0Z5h56EEJIVDg7ccwxnXmvcT0OC8nyWVYW7zhmuNaR2UXGEdFMAu8b/mJSV9aMY/Gvlr3lnHzl7UffteveZ533v4/+m2G6ZkoUXA6KTy+c1ePADGbGlVqRpWzFY4lFzGCYcKCCcZwLwf+IGjC7UB8K+lVaVaUhlmJdksYN4P/ICAAsAx+XkBSgpZwLwf+AElHgBwmfy8gLbVVEp6b55PXPz7bTWVjF7B1QgoAOBCzPuB11HiAQCXYt4PvIyAAgAuxrwfeBUlHgAAYB0CCgAAsA4lHgAAfKJ/wLhmzhIBBQAAHzjU1qkdB9vVGX1vheHSUIG21VRaedcXJR4AADwuvl/W+8OJ9N7u14faOnPUspERUAAA8LBr7X4tDe5+3T9g197BBBQAADzseMelISMn72ckdUZ7dLzjUvYaNQoEFAAAPMytu18zSRbWcdMscwCwnVt3vyagwCpum2UOALZz6+7XlHhgDTfOMgcA27l192sCCqzg1lnmAOAGbtz9mhIPrJDKLHM2RgOA1Llt92sCCqzg1lnmAOAmbtr9mhIPrODWWeYAgMwgoMAK8VnmIw00BjR4N49ts8wBAJlBQIEV3DrLHACQGQQUWMONs8wBAJnBJFlYxW2zzAEAmUFAgXXcNMsc9mGrBMAbCCgAPIOtEgDvYA4KAE9gqwTAWwgowCj0DxgdO3tRz7de0LGzF1ly3zJslQB4DyUe4BooG9iPrRIA72EEBbgKygbuwFYJgPcQUIARUDZwD7ZKALyHgAKMIJWyAXKLrRIA7yGgACOgbOAebJUAeA8BBRgBZQN3YasEwFu4iwcYQbxsEIn2DDsPJaDBDz/KBvZgqwTAOwgowAjiZYPVjS0KSEkhhbKBvdgqAfAGSjzAVVA2AIDcYAQFuAbKBgCQfQQUYBQoGwBAdlHiAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKzjeEB599139dWvflXl5eWaOHGiZs2apccee0wDAwOJc4wx2r59u8rKyjRx4kQtXrxYZ86ccbopAADApRwPKDt37tS3v/1t7dmzR//93/+tXbt26e/+7u/0jW98I3HOrl27tHv3bu3Zs0cnTpxQOBzWkiVL1N3d7XRzAACACzkeUI4dO6bPfe5zWrZsmT784Q/rz/7sz1RdXa2f/exnkgZHT5588klt3bpV99xzj6qqqvT000/rnXfe0f79+51uDgAAcCHHA8rChQv1k5/8RK+//rok6ec//7mOHDmiT3/605Kkjo4ORSIRVVdXJ/5NMBjUokWLdPToUaebAwAAXGic0z9w06ZNikajuvHGG5Wfn6/+/n49/vjjuu+++yRJkUhEklRSUpL070pKSvTmm28O+zN7e3vV29ub+D4WizndbAAAYBHHR1CeeeYZNTY2av/+/WppadHTTz+tv//7v9fTTz+ddF4gEEj63hgz5FhcfX29QqFQ4mvGjBlONxsAAFjE8YDyla98RZs3b9YXvvAF3XLLLXrggQf06KOPqr6+XpIUDoclvTeSEtfV1TVkVCVuy5Ytikajia/z58873WwAAGARxwPKO++8o7y85B+bn5+fuM24vLxc4XBYTU1Nicf7+vrU3NysBQsWDPszg8GgioqKkr4AAIB3OT4HpaamRo8//riuv/563XzzzTp16pR2796tL33pS5IGSzu1tbWqq6tTRUWFKioqVFdXp0mTJmn58uVONwcAALiQ4wHlG9/4hv7mb/5Ga9asUVdXl8rKyrRy5Ur97d/+beKcjRs36sqVK1qzZo0uX76s+fPn6/DhwyosLHS6OQAAwIUCxhiT60akKhaLKRQKKRqNUu4BAMAlUvn8Zi8eAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrOL4OCgBcTf+A0fGOS+rq7tG0wgLNKy9Wft7w+3AB8C8CCoCsOdTWqR0H29UZ7UkcKw0VaFtNpZZWleawZQBsQ4kHQFYcauvU6saWpHAiSZFoj1Y3tuhQW2eOWgbARgQUABnXP2C042C7hlu2On5sx8F29Q+4bmFrwAr9A0bHzl7U860XdOzsRU+8lijxAMi44x2XhoycvJ+R1Bnt0fGOS7rthinZaxjgAV4tnTKCAiDjurpHDifpnAdgkJdLpwQUABk3rbDA0fMAeL90SkBxgBdrf4CT5pUXqzRUoJFuJg5ocEh6XnlxNpsFuFoqpVM3Yg7KGHm19gc4KT8voG01lVrd2KKAlPQXXzy0bKupZD0UIAVeL50ygjIGXq79AU5bWlWqhhWzFQ4ll3HCoQI1rJhNoAdS5PXSKSMoabpW7S+gwdrfksowfxUiwe+rqC6tKtWSyrCv+wBwSrx0Gon2DPtZFNDgHwBuLZ0SUNLEbZNIFeXAQfl5AV4TgAO8XjqlxJMmr9f+4CzKgQAywculU0ZQ0uT12h+cQzkQwFhcqzTs1dIpASVNXq/9wTmUAwGka7SlYS+WTinxpCle+5M0ZG0HL9T+4BzKgQDS4ffSMAFlDLxc+4NzKAcCSJXXV4kdDUo8Y+TV2h+cQzkQQKpSLQ17cQkDAooDvFj7g3O8fisgAOelUhr26hIGlHiALKAcCCAVoy35/vq373h2nkrAGOO6AlYsFlMoFFI0GlVRUVGum+NaXhwStB197m78/pAt/QNGC3e+dNXScElRUFJAkdjwoy3x8vGRTXdY8zxN5fObEo9PeXVI0HaUA92L1wyyaTSl4fvmXa+v//iXI/4Mty9hQInHh/x+6xqQKl4zyIVrlYY/PHXyqH6OW5cwYATFZ1jVFEgNrxnk0tXuFD129uKofoZblzAgoPgMq5oCqeE1g1wbqTTs9SUMKPH4DKuaAqnhNQNbeX1FcwKKz7CqKZAaXjOwmZeXMKDE4zNeHxIEnMZrBrbz6ormjKD4jNeHBAGn8ZqBG8TnqXzuY9fpthumeOL5SEDxIS8PCQKZwGsGyD5WkvUxVsUEUsNrBhgbVpLFqLCqKZAaXjNA9lDiAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHXG5boBeE//gNHxjkvq6u7RtMICzSsvVn5eINfNwhj47Xfqt+sFkDkZCSgXLlzQpk2b9KMf/UhXrlzRH//xH+s73/mO5syZI0kyxmjHjh3au3evLl++rPnz5+ub3/ymbr755kw0xxUOtXVqx8F2dUZ7EsdKQwXaVlOppVWlOWwZ0uW336nfrhdAZjle4rl8+bJuv/12jR8/Xj/60Y/U3t6uf/iHf9Af/uEfJs7ZtWuXdu/erT179ujEiRMKh8NasmSJuru7nW6OKxxq69TqxpakN3ZJikR7tLqxRYfaOnPUMqTLb79Tv10vgMwLGGOMkz9w8+bN+s///E+98sorwz5ujFFZWZlqa2u1adMmSVJvb69KSkq0c+dOrVy58pr/RywWUygUUjQaVVFRkZPNz7r+AaOFO18a8sYeF5AUDhXoyKY7GCp3Cb/9Tv12vQDSl8rnt+MjKC+88ILmzp2rz3/+85o2bZpuvfVW7du3L/F4R0eHIpGIqqurE8eCwaAWLVqko0ePDvsze3t7FYvFkr684njHpRHf2CXJSOqM9uh4x6XsNQpj4rffqd+uF0B2OB5QfvWrX6mhoUEVFRX693//d61atUpf/vKX9c///M+SpEgkIkkqKSlJ+nclJSWJxz6ovr5eoVAo8TVjxgynm50zXd0jv7Gncx5yz2+/U79dL4DscDygDAwMaPbs2aqrq9Ott96qlStX6uGHH1ZDQ0PSeYFA8lCvMWbIsbgtW7YoGo0mvs6fP+90s3NmWmGBo+ch9/z2O/Xb9QLIDscDSmlpqSorK5OO3XTTTTp37pwkKRwOS9KQ0ZKurq4hoypxwWBQRUVFSV9eMa+8WKWhAo1UmQ9o8E6IeeXF2WwWxsBvv1O/XS+A7HA8oNx+++167bXXko69/vrrmjlzpiSpvLxc4XBYTU1Nicf7+vrU3NysBQsWON0c6+XnBbStZjDQffANPv79tppKJhe6iN9+p367XgDZ4XhAefTRR/Xqq6+qrq5Ob7zxhvbv36+9e/dq7dq1kgZLO7W1taqrq9Nzzz2ntrY2PfTQQ5o0aZKWL1/udHNcYWlVqRpWzFY4lDwEHg4VqGHFbNaQcCG//U79dr0AMs/x24wl6Yc//KG2bNmiX/7ylyovL9eGDRv08MMPJx6PL9T2T//0T0kLtVVVVY3q53vpNuP3YxVO7/Hb79Rv1wsgNal8fmckoGSaVwMKAABeltN1UAAAAMaKgAIAAKxDQAEAANYhoAAAAOsQUAAAgHXG5boBAAA7cds4comAAgAY4lBbp3YcbE/aqbo0VKBtNZUsvIesoMQDAEhyqK1TqxtbksKJJEWiPVrd2KJDbZ05ahn8hIACAEjoHzDacbBdw63gGT+242C7+gdct8YnXIaAAgBION5xacjIyfsZSZ3RHh3vuJS9RsGXCCgAgISu7pHDSTrnAekioAAAEqYVFlz7pBTOA9JFQAEAJMwrL1ZpqEAj3Uwc0ODdPPPKi7PZLPgQAQUAkJCfF9C2mkpJGhJS4t9vq6lkPRRkHAEFAJBkaVWpGlbMVjiUXMYJhwrUsGI266AgK1ioDQAwxNKqUi2pDLOSLHKGgAIAGFZ+XkC33TAl181whNeX7ffi9RFQAACe5vVl+716fcxBAQB4lteX7ffy9RFQAACe5PVl+71+fQQUAIAneX3Zfq9fHwEFAOBJXl+23+vXR0ABAHiS15ft9/r1EVAAAJ7k9WX7vX59BBQAgCd5fdl+r18fAQUA4FleX7bfy9cXMMa47v6jWCymUCikaDSqoqKiXDcHAGA5L660+n5uub5UPr9ZSRYA4HleWrZ/OF68Pko8AADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsw108QI655fZAAMgmAgqQQ4faOrXjYHvSjqSloQJtq6l09QJLADBWlHiAHDnU1qnVjS1DtkuPRHu0urFFh9o6c9QyAMg9AgqQA/0DRjsOtmu4ZZzjx3YcbFf/gOsWegas1D9gdOzsRT3fekHHzl7kteUClHiAHDjecWnIyMn7GUmd0R4d77jkudUhgWyjlOpOjKAAOdDVPXI4Sec8AMOjlOpeBBQgB6YVFlz7pBTOAzAUpVR3I6C4ELVU95tXXqzSUIFGupk4oMEh6HnlxdlsFuApqZRSYR/moLgMtVRvyM8LaFtNpVY3tiggJf2FFw8t22oqWQ8FGANKqe7GCIqLUEv1lqVVpWpYMVvhUHIZJxwqUMOK2QROYIwopbobIyguca1aakCDtdQllWH+6s4Cp1Z/XVpVqiWVYVaSHQVW3EWq4qXUSLRn2PfOgAb/IKCUaicCiktwW6o9nC6z5ecF+J1dA6VNpINSqrtR4nEJaql2oMyWffQ5xoJSqnsxguIS1FJzjzJb9tHncAKlVHcioLgEtdTco8yWffQ5nEIp1X0o8bhEvJYqacjaGdRSs4MyW/bR54B/EVBchFpqblFmyz76HPAvSjwuQy01dyizZR99nh5uyYYXEFBciFpqbnDLYvbR56njlmx4BSUeIAWU2bKPPh89bsmGlwSMMa7baS4WiykUCikajaqoqCjXzYEPMYSefV7q80xcS/+A0cKdL41411O8HHZk0x2u7bdUeOn54iWpfH5T4gHSQJkt+7zS55kqwXBL9nsoc3kDJR4AyJJMlmC4JXsQZS7vIKAAQBZca1VcaXBV3P6B9Kru3JKd+T5GdhFQACALUinBpCN+S/ZIsywCGixzePmW7Ez3MbIr4wGlvr5egUBAtbW1iWPGGG3fvl1lZWWaOHGiFi9erDNnzmS6KQCQM5kuwbDaNGUur8loQDlx4oT27t2rj3zkI0nHd+3apd27d2vPnj06ceKEwuGwlixZou7u7kw2BwByJhslGL/fkk2Zy1sydhfP22+/rfvvv1/79u3T1772tcRxY4yefPJJbd26Vffcc48k6emnn1ZJSYn279+vlStXZqpJAJAz2VoV18+rTbPysLdkbARl7dq1WrZsme66666k4x0dHYpEIqqurk4cCwaDWrRokY4ePTrsz+rt7VUsFkv6AgA3yWYJJn5L9uc+dp1uu2GKL8KJRJnLazISUA4cOKCWlhbV19cPeSwSiUiSSkpKko6XlJQkHvug+vp6hUKhxNeMGTOcbzQAZJjfSzDZQB97h+MlnvPnz2v9+vU6fPiwCgpGrvMFAskJ1hgz5Fjcli1btGHDhsT3sViMkALAlfxcgskW+tgbHA8oJ0+eVFdXl+bMmZM41t/fr5dffll79uzRa6+9JmlwJKW09L0k29XVNWRUJS4YDCoYDDrdVADICa+simsz+tj9HC/x3HnnnTp9+rRaW1sTX3PnztX999+v1tZWzZo1S+FwWE1NTYl/09fXp+bmZi1YsMDp5gAAABdyfASlsLBQVVVVSccmT56sKVOmJI7X1taqrq5OFRUVqqioUF1dnSZNmqTly5c73RwAAOBCOdkscOPGjbpy5YrWrFmjy5cva/78+Tp8+LAKCwtz0RwAAGCZgDHGdZsSpLJdMwAAsEMqn9/sxQMAAKyTkxIPAACj0T9gXHm7sFvbbRMCCgDASofaOrXjYHvSDsWloQJtq6m0esE1t7bbNpR4AADWOdTWqdWNLUkf8pIUifZodWOLDrV15qhlV+fWdtuIgAIAsEr/gNGOg+3DbvgXP7bjYLv6B+y6x8Ot7bYVAcUi/QNGx85e1POtF3Ts7EWexAB86XjHpSEjEO9nJHVGe3S841L2GjUKbm23rZiDYglqlgAwqKt75A/5dM7LFre221aMoFiAmiUAvGda4cgbzaZzXra4td22IqDkGDVLAEg2r7xYpaECjXRTbkCDI8zzyouz2axrcmu7bUVAyTFqlgCQLD8voG01lZI05MM+/v22mkrr1hVxa7ttRUDJMWqWADDU0qpSNayYrXAouRwSDhWoYcVsa+fmubXdNmKSbI5RswSA4S2tKtWSyrDrVmR1a7ttQ0DJsXjNMhLtGXYeSkCDyZuaJQA/ys8L6LYbpuS6GSlza7ttQoknx6hZAgAwFAHFAtQsAQBIRonHEtQsAQB4DwHFItQsAQAYRIkHAABYh4ACAACsQ4kHSEP/gGG+EABkEAEFSBE7TwNA5lHiAVLAztMAkB0EFGCU2HkagB/0DxgdO3tRz7de0LGzF3P2nkaJBxilVHae5nZxAG5kUwmbERRglNh5GoCX2VbCJqAAo8TO0wC8ysYSNgEFGKX4ztMj3Uwc0OBQKDtPA3CbVErY2UJAAUaJnacBeJWNJWwCCpACdp4G4EU2lrC5iwdIETtPA/CaeAk7Eu0Zdh5KQIN/iGWzhE1AAdLAztMAvCRewl7d2KKAlBRSclXCpsQDAACsK2EzggIAACTZVcImoAAAgARbStiUeAAAgHUIKAAAwDqUeJAx/QPGijomAG/jvcabCCjICJt2xATgXbzXeBclHjjOth0xAXgT7zXeRkCBo2zcEROA9/Be430EFDjKxh0xAXgP7zXeR0CBo2zcEROA9/Be430EFDjKxh0xAXgP7zXeR0CBo+I7Yo50g19AgzPss7kjJgDv4b3G+wgocFR8R0xJQ944crUjJgDv4b3G+wgocJxtO2IC8Cbea7wtYIxx3T1YsVhMoVBI0WhURUVFjv1cViN0Fv0JIBt4r3GPVD6/WUn2/7AaofNs2RETgLfxXuNNlHjEaoQAANjG9yMo11qNMKDB1QiXVIYZMnSZdId9GS4GgNzzfUBJZTVChhDdI92SHaU+ALCD70s8rEboPemW7Cj1AYA9fB9QWI3QW9LdQIyNxwDALr4PKKxG6C3pbiDGxmO51T9gdOzsRT3fekHHzl4kCAJgDkp8NcLVjS0KSEl/QbMaofukW7Kj1Jc7zPsBMBzfj6BIrEboJemW7Cj15QbzfgCMxPcjKHFLq0q1pDLM7aUuFy/ZRaI9w84nCWgweH6wZJfuv0P6uMUfwNU4PoJSX1+vj3/84yosLNS0adN0991367XXXks6xxij7du3q6ysTBMnTtTixYt15swZp5uSsvhqhJ/72HW67YYpvCm6ULobiLHxWPYx7wfA1TgeUJqbm7V27Vq9+uqrampq0rvvvqvq6mr97ne/S5yza9cu7d69W3v27NGJEycUDoe1ZMkSdXd3O90c+FC6JTtKfdnFvB8AV5PxzQL/53/+R9OmTVNzc7M+8YlPyBijsrIy1dbWatOmTZKk3t5elZSUaOfOnVq5cuU1f2amNguEt7CSrN2Onb2o+/a9es3zvvfw/2ORRMAjrNosMBqNSpKKiwdr9x0dHYpEIqqurk6cEwwGtWjRIh09enTYgNLb26ve3t7E97FYLMOthheku4EYG49lB/N+AFxNRu/iMcZow4YNWrhwoaqqqiRJkUhEklRSUpJ0bklJSeKxD6qvr1coFEp8zZgxI5PNBpAFzPsBcDUZDSjr1q3TL37xC33ve98b8lggkPymY4wZcixuy5Ytikajia/z589npL0Asot5PwBGkrESzyOPPKIXXnhBL7/8sqZPn544Hg6HJQ2OpJSWvvfm09XVNWRUJS4YDCoYDGaqqQByiFv8AQzH8REUY4zWrVunZ599Vi+99JLKy8uTHi8vL1c4HFZTU1PiWF9fn5qbm7VgwQKnmwPABbjFH8AHOT6CsnbtWu3fv1/PP/+8CgsLE/NKQqGQJk6cqEAgoNraWtXV1amiokIVFRWqq6vTpEmTtHz5cqebAwAAXMjxgNLQ0CBJWrx4cdLxp556Sg899JAkaePGjbpy5YrWrFmjy5cva/78+Tp8+LAKCwudbg4AAHChjK+DkgmsgwIAgPuk8vnNZoEAAMA6BBQAAGAddjNGylgKHgCQaQQUpORQW6d2HGxP2oW2NFSgbTWVLKoFAHAMJR6M2qG2Tq1ubEkKJ5IUifZodWOLDrV15qhlAACvIaBgVPoHjHYcbB92U7f4sR0H29U/4LqbwgAAFiKgYFSOd1waMnLyfkZSZ7RHxzsuZa9RAADPIqBgVLq6Rw4n6ZwHAMDVEFAwKtMKC659UgrnAQBwNQQUjMq88mKVhgo00s3EAQ3ezTOvvDibzQIAeBQBBaOSnxfQtppKSRoSUuLfb6upZD0UAIAjCCgYtaVVpWpYMVvhUHIZJxwqUMOK2ayDAgBwDAu1ISVLq0q1pDLMSrIAgIwioCBl+XkB3XbDlFw3AwDgYZR4AACAdQgoAADAOpR4cE3sXgxkHq8zpMIPzxcCCq6K3YuBzON1hlT45flCiQcjYvdiIPN4nSEVfnq+EFAwLHYvBjKP1xlS4bfnCwEFw2L3YiDzeJ0hFX57vhBQMCx2LwYyj9cZUuG35wsBBcNi92Ig83idIRV+e74QUDAsdi8GMo/XGVLht+cLAQXDYvdiIPN4nSEVfnu+EFAwInYvBjKP1xlS4afnS8AY47r7kWKxmEKhkKLRqIqKinLdHM/zw4qFQK7xOkMq3Pp8SeXzm5VkcU3sXgxkHq8zpMIPzxdKPAAAwDqMoAAYFbcOKQNwJwIKgGvyy+ZkAOxBiQfAVflpczIA9iCgABiR3zYnA2APAgqs0z9gdOzsRT3fekHHzl7kwy+H/LY5GQB7MAcFVmGug138tjkZAHswggJrMNfBPn7bnAyAPQgosAJzHezkt83JANiDgAIrMNfBTn7bnAyAPQgosAJzHezlp83JANiDSbKwAnMd7La0qlRLKsOsJAsgawgosEJ8rkMk2jPsPJSABv9iZ65D7vhhczIA9qDEAysw1wEA8H4EFFiDuQ4AgDhKPLAKcx0AABIBBRZirgMAgBIPAACwDgEFAABYhxIPAMDV+gcM89ZGyU19RUABALgWO6CPntv6ihIPAMCV2AF99NzYVwSU9+kfMDp29qKeb72gY2cven7nXL9dLwDvYAf00XNrX1Hi+T9uG/oaK79dLwBvSWUHdL8vW+DWvmIERe4c+hoLv10vAO9hB/TRc2tf+T6guHXoK11+u14A3pTqDuh+Lmm7dbd435d43Dr0lS6/XS8Ab0plB3S/l7Tdulu870dQ3Dr0lS6/XS8AbxrtDuhN7RHfl7Tdulu87wOKW4e+0uW36wXgXdfaAX1JZZiS9v9x427xvi/xuHXoK11+u14A3na1HdCPnb1ISft93LZbfE5HUL71rW+pvLxcBQUFmjNnjl555ZWst8GtQ1/p8tv1AvC++A7on/vYdbrthimJ9y9K2kON1Fc2yllAeeaZZ1RbW6utW7fq1KlT+pM/+RP96Z/+qc6dO5f1trhx6Gss/Ha9APyJkra7BYwxOSm+zZ8/X7Nnz1ZDQ0Pi2E033aS7775b9fX1V/23sVhMoVBI0WhURUVFjrXJTZsoOcFv1wvAX/oHjBbufOmaJe0jm+7gvS9LUvn8zskclL6+Pp08eVKbN29OOl5dXa2jR48OOb+3t1e9vb2J72OxWEbaFR/68gu/XS8Af4mXtFc3tiggJYUUStr2y0mJ57e//a36+/tVUlKSdLykpESRSGTI+fX19QqFQomvGTNmZKupAAAXo6TtXjm9iycQSE6txpghxyRpy5Yt2rBhQ+L7WCxGSAEAjIrb7l7BoJwElKlTpyo/P3/IaElXV9eQURVJCgaDCgaD2WoeAMBjKGm7T05KPBMmTNCcOXPU1NSUdLypqUkLFizIRZMAAIBFclbi2bBhgx544AHNnTtXt912m/bu3atz585p1apVuWoSAACwRM4Cyr333quLFy/qscceU2dnp6qqqvTiiy9q5syZuWoSAACwRM7WQRmLTK2DAgAAMieVz2/fbxYIAADsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6OV3qPl3xG48ytWkgAABwXvxzezQ3ELsyoHR3d0sS+/EAAOBC3d3dCoVCVz3HleugDAwM6K233lJhYeGwmwuOVnzTwfPnz7OeSpbQ59lHn2cffZ599Hn2pdPnxhh1d3errKxMeXlXn2XiyhGUvLw8TZ8+3bGfV1RUxBM6y+jz7KPPs48+zz76PPtS7fNrjZzEMUkWAABYh4ACAACs4+uAEgwGtW3bNgWDwVw3xTfo8+yjz7OPPs8++jz7Mt3nrpwkCwAAvM3XIygAAMBOBBQAAGAdAgoAALAOAQUAAFjH1wHlW9/6lsrLy1VQUKA5c+bolVdeyXWTPKG+vl4f//jHVVhYqGnTpunuu+/Wa6+9lnSOMUbbt29XWVmZJk6cqMWLF+vMmTM5arH31NfXKxAIqLa2NnGMPnfehQsXtGLFCk2ZMkWTJk3Sxz72MZ08eTLxOH3urHfffVdf/epXVV5erokTJ2rWrFl67LHHNDAwkDiHPh+bl19+WTU1NSorK1MgENAPfvCDpMdH07+9vb165JFHNHXqVE2ePFmf/exn9Zvf/Cb1xhifOnDggBk/frzZt2+faW9vN+vXrzeTJ082b775Zq6b5nqf+tSnzFNPPWXa2tpMa2urWbZsmbn++uvN22+/nTjniSeeMIWFheb73/++OX36tLn33ntNaWmpicViOWy5Nxw/ftx8+MMfNh/5yEfM+vXrE8fpc2ddunTJzJw50zz00EPmv/7rv0xHR4f58Y9/bN54443EOfS5s772ta+ZKVOmmB/+8Iemo6PD/Nu//Zv5gz/4A/Pkk08mzqHPx+bFF180W7duNd///veNJPPcc88lPT6a/l21apW57rrrTFNTk2lpaTGf/OQnzUc/+lHz7rvvptQW3waUefPmmVWrViUdu/HGG83mzZtz1CLv6urqMpJMc3OzMcaYgYEBEw6HzRNPPJE4p6enx4RCIfPtb387V830hO7ublNRUWGamprMokWLEgGFPnfepk2bzMKFC0d8nD533rJly8yXvvSlpGP33HOPWbFihTGGPnfaBwPKaPr3f//3f8348ePNgQMHEudcuHDB5OXlmUOHDqX0//uyxNPX16eTJ0+quro66Xh1dbWOHj2ao1Z5VzQalSQVFxdLkjo6OhSJRJL6PxgMatGiRfT/GK1du1bLli3TXXfdlXScPnfeCy+8oLlz5+rzn/+8pk2bpltvvVX79u1LPE6fO2/hwoX6yU9+otdff12S9POf/1xHjhzRpz/9aUn0eaaNpn9Pnjyp3//+90nnlJWVqaqqKuXfgSs3Cxyr3/72t+rv71dJSUnS8ZKSEkUikRy1ypuMMdqwYYMWLlyoqqoqSUr08XD9/+abb2a9jV5x4MABtbS06MSJE0Meo8+d96tf/UoNDQ3asGGD/vqv/1rHjx/Xl7/8ZQWDQX3xi1+kzzNg06ZNikajuvHGG5Wfn6/+/n49/vjjuu+++yTxPM+00fRvJBLRhAkT9KEPfWjIOal+vvoyoMQFAoGk740xQ45hbNatW6df/OIXOnLkyJDH6H/nnD9/XuvXr9fhw4dVUFAw4nn0uXMGBgY0d+5c1dXVSZJuvfVWnTlzRg0NDfriF7+YOI8+d84zzzyjxsZG7d+/XzfffLNaW1tVW1ursrIyPfjgg4nz6PPMSqd/0/kd+LLEM3XqVOXn5w9Jc11dXUOSIdL3yCOP6IUXXtBPf/pTTZ8+PXE8HA5LEv3voJMnT6qrq0tz5szRuHHjNG7cODU3N+sf//EfNW7cuES/0ufOKS0tVWVlZdKxm266SefOnZPE8zwTvvKVr2jz5s36whe+oFtuuUUPPPCAHn30UdXX10uizzNtNP0bDofV19eny5cvj3jOaPkyoEyYMEFz5sxRU1NT0vGmpiYtWLAgR63yDmOM1q1bp2effVYvvfSSysvLkx4vLy9XOBxO6v++vj41NzfT/2m68847dfr0abW2tia+5s6dq/vvv1+tra2aNWsWfe6w22+/fcjt86+//rpmzpwpied5JrzzzjvKy0v+2MrPz0/cZkyfZ9Zo+nfOnDkaP3580jmdnZ1qa2tL/XeQ1tReD4jfZvyd73zHtLe3m9raWjN58mTz61//OtdNc73Vq1ebUChk/uM//sN0dnYmvt55553EOU888YQJhULm2WefNadPnzb33XcftwI67P138RhDnzvt+PHjZty4cebxxx83v/zlL82//uu/mkmTJpnGxsbEOfS5sx588EFz3XXXJW4zfvbZZ83UqVPNxo0bE+fQ52PT3d1tTp06ZU6dOmUkmd27d5tTp04lluAYTf+uWrXKTJ8+3fz4xz82LS0t5o477uA241R985vfNDNnzjQTJkwws2fPTtwGi7GRNOzXU089lThnYGDAbNu2zYTDYRMMBs0nPvEJc/r06dw12oM+GFDoc+cdPHjQVFVVmWAwaG688Uazd+/epMfpc2fFYjGzfv16c/3115uCggIza9Yss3XrVtPb25s4hz4fm5/+9KfDvn8/+OCDxpjR9e+VK1fMunXrTHFxsZk4caL5zGc+Y86dO5dyWwLGGJP2eA8AAEAG+HIOCgAAsBsBBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADW+f9+RQ1eSnJHogAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "np.random.seed(888)\n",
    "X = np.random.randint(0,100,(50,2)) # 生成一个50×2的矩阵，每一个元素在0~100随机生成整数\n",
    "plt.scatter(X[:,0], X[:,1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "8b0779e7-3ee8-4a80-8ade-8f6c7bc67e12",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGdCAYAAADaPpOnAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAMGRJREFUeJzt3X90VPWd//HXEHGiNRkXYjKDjRCtojFdhbBIWH/7JQa2WUWPR2tlcc8uZ8GyfpX1VNE9hbTHpvaoZT1UWHvYKhtrOWfxFwc2x+yBgC2hgCRVGqWWjSYHZxoJOhNtEzS53z/ynaljksnMMHfmfu48H+fMOeTmM+R9P7kz887n/bmfj8eyLEsAAACGmJTrAAAAAFJB8gIAAIxC8gIAAIxC8gIAAIxC8gIAAIxC8gIAAIxC8gIAAIxC8gIAAIxyWq4DyLTh4WF98MEHKioqksfjyXU4AAAgCZZlqb+/X9OmTdOkSYnHVlyXvHzwwQcqLy/PdRgAACANPT09+upXv5qwjeuSl6KiIkkjJ19cXJzjaAAAQDIikYjKy8tjn+OJuC55iZaKiouLSV4AADBMMlM+mLALAACMQvICAACMQvICAACMYmvy0tjYqL/6q79SUVGRSktLdfPNN+vIkSMTPm/37t2qrq5WYWGhzj//fG3cuNHOMAEAgEFsTV52796tb3/729q3b59aWlr0+eefq7a2Vp9++um4z+nq6tKiRYt01VVXqb29XQ8//LDuvfdebd261c5QAQCAITyWZVnZ+mEffvihSktLtXv3bl199dVjtnnwwQf16quv6u23344dW758uX7zm9+ora1twp8RiUTk8/kUDoe52wgAAEOk8vmd1Tkv4XBYkjRlypRx27S1tam2tjbu2I033qiDBw/qs88+G9V+cHBQkUgk7gEAANwra8mLZVlatWqVrrzySlVVVY3bLhQKqaysLO5YWVmZPv/8cx0/fnxU+8bGRvl8vtiD1XUBAHC3rCUvK1eu1JtvvqkXXnhhwrZfXqAmWtkaa+Ga1atXKxwOxx49PT2ZCRhARg0NW2o72qdXOo6p7WifhoazVrEG4DJZWWH3n//5n/Xqq69qz549E+5X4Pf7FQqF4o719vbqtNNO09SpU0e193q98nq9GY0XQGY1Hw6qYVunguGB2LGAr1Br6itVVxXIYWQATGTryItlWVq5cqVefPFF7dy5UxUVFRM+p6amRi0tLXHHXnvtNc2ZM0eTJ0+2K1QANmk+HNSKpkNxiYskhcIDWtF0SM2HgzmKDICpbE1evv3tb6upqUk///nPVVRUpFAopFAopD/96U+xNqtXr9bf/d3fxb5evny53n//fa1atUpvv/22/uM//kObNm3SAw88YGeoAGwwNGypYVunxioQRY81bOukhAQgJbYmLxs2bFA4HNa1116rQCAQe2zZsiXWJhgMqru7O/Z1RUWFduzYodbWVl1++eX6/ve/r6eeekq33nqrnaECsMH+rhOjRly+yJIUDA9of9eJ7AWVR5hnBLeydc5LMkvIPPvss6OOXXPNNTp06JANEQHIpt7+8ROXdNohecwzgpuxtxEA25QWFWa0HZLDPCO4HckLANvMrZiigK9Qoxc5GOHRyGjA3IrxF65EaphnhHxA8gLANgWTPFpTXylJoxKY6Ndr6itVMGm89AapYp4R8gHJCwBb1VUFtOGu2fL74ktDfl+hNtw1m/kXGcY8I+SDrCxSByC/1VUFtKDSr/1dJ9TbP6DSopFSESMumcc8I+QDkhcAWVEwyaOaC0avko3Mis4zCoUHxpz34tHIqBfzjGAyykYA4CLMM0I+IHkBAJdhnhHcjrIRALgQ84zgZiQvAOBSzDOCW1E2AgAARiF5AQAARqFsBAAANDRsGTNHiuQFAIA8Z9ou5JSNAADIYybuQk7yAgBAnjJ1F3KSFwAA8pSpu5CTvAAAkKdM3YWcCbswikmz4QHA6UzdhZzkBcYwbTY8ADidqbuQUzaCEUycDQ8ATmfqLuQkL3A8U2fDA4AJTNyFnLIRHC+V2fBsQgcAqTNtF3KSFzieqbPhAcAkJu1CTtkIjmfqbHgAgD1IXuB40dnw4w1eejRy15HTZsMDAOxB8gLHM3U2PADAHiQvMIKJs+EBAPZgwi6MYdpseACAPUheYBSTZsPDedheAnAHkhcAeYHtJQD3YM4LANdjewnAXUhegFM0NGyp7WifXuk4prajfWxT4DBsLwG4D2Uj4BRQinA+tpcA3IeRFyBNlCLMwPYSgPuQvABpoBRhDraXANyH5AVIQyqlCOQW20sA7mNr8rJnzx7V19dr2rRp8ng8evnllxO2b21tlcfjGfV455137AwTSBmlCHOwvQTgPrYmL59++qkuu+wyrV+/PqXnHTlyRMFgMPa48MILbYoQSA+lCLOwvQTgLrbebbRw4UItXLgw5eeVlpbq7LPPznxAQIZESxGh8MCY8148GvlgpBThHGwvAbiHI+e8zJo1S4FAQDfccIN27dqVsO3g4KAikUjcA7AbpQgzRbeXuOnyc1VzwVR+P4ChHJW8BAIBPfPMM9q6datefPFFzZw5UzfccIP27Nkz7nMaGxvl8/lij/Ly8ixGjHxGKQIAcsNjWVZW7uX0eDx66aWXdPPNN6f0vPr6enk8Hr366qtjfn9wcFCDg4OxryORiMrLyxUOh1VcXHwqIQNJYbM/ADh1kUhEPp8vqc9vx6+wO2/ePDU1NY37fa/XK6/Xm8WIgHjsdA0A2eWostFY2tvbFQgw/A4AAEbYOvLyySef6Pe//33s666uLnV0dGjKlCk677zztHr1ah07dkybN2+WJK1bt04zZszQpZdeqpMnT6qpqUlbt27V1q1b7QwTAAAYxNbk5eDBg7ruuutiX69atUqStHTpUj377LMKBoPq7u6Off/kyZN64IEHdOzYMZ1xxhm69NJLtX37di1atMjOMAEAgEGyNmE3W1KZ8AMAAJwhlc9vx895AQAA+CKSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYJTTch0AAEQNDVva33VCvf0DKi0q1NyKKSqY5Ml1WAAchuQFgCM0Hw6qYVunguGB2LGAr1Br6itVVxXIYWQAnIayEYCcaz4c1IqmQ3GJiySFwgNa0XRIzYeDOYoMgBORvADIqaFhSw3bOmWN8b3osYZtnRoaHqsFgIkMDVtqO9qnVzqOqe1onyteS5SNAOTU/q4To0ZcvsiSFAwPaH/XCdVcMDV7gQEu4NZyLCMvAHKqt3/8xCWddgBGuLkcS/ICIKdKiwoz2g6A+8uxJC82c2OtEcikuRVTFPAVarwboj0aGeaeWzElm2EBRkulHGsi5rzYyK21RiCTCiZ5tKa+UiuaDskjxf2lGE1o1tRXst4LkAK3l2NtHXnZs2eP6uvrNW3aNHk8Hr388ssTPmf37t2qrq5WYWGhzj//fG3cuNHOEG3j5lojkGl1VQFtuGu2/L740pDfV6gNd80m2QdS5PZyrK0jL59++qkuu+wy/f3f/71uvfXWCdt3dXVp0aJFWrZsmZqamvSrX/1K99xzj84555yknu8UE9UaPRqpNS6o9PPXJGLyfXXZuqqAFlT687oPgEyJlmND4YExP4s8GvnjwNRyrK3Jy8KFC7Vw4cKk22/cuFHnnXee1q1bJ0m65JJLdPDgQT3++ONGJS/c+olUUWIcUTDJw2sCyAC3l2MdNWG3ra1NtbW1ccduvPFGHTx4UJ999tmYzxkcHFQkEol75Jrba43ILEqMAOzg5nKsoybshkIhlZWVxR0rKyvT559/ruPHjysQGN3RjY2NamhoyFaISXF7rRGZQ4kRwKmYqNzs1nKso5IXSfJ44jvUsqwxj0etXr1aq1atin0diURUXl5uX4BJcHutEZlDiRFAupItN7uxHOuospHf71coFIo71tvbq9NOO01Tp47d8V6vV8XFxXGPXIvWGiWNWrvCDbVGZA4lRgDpyPdys6OSl5qaGrW0tMQde+211zRnzhxNnjw5R1Glx821RmQOJUYAqXL76rnJsLVs9Mknn+j3v/997Ouuri51dHRoypQpOu+887R69WodO3ZMmzdvliQtX75c69ev16pVq7Rs2TK1tbVp06ZNeuGFF+wM0zZurTUicygxAkhVquVmNy7DYGvycvDgQV133XWxr6NzU5YuXapnn31WwWBQ3d3dse9XVFRox44duv/++/WTn/xE06ZN01NPPWXUbdJf5sZaIzLH7bczAsi8VMrNbl2GwWNFZ8S6RCQSkc/nUzgcdsT8FyAZbn2DAZB5bUf79M2f7puw3f3/5yKt+5/fjRrVjf4p5LQpDKl8fpO8YExuHGZ0OvrcbPz+kC1Dw5aufGxnwnJzWbFXkkehyNijNNGS9C8fvN4x12kqn9+Ou1UauccoQG5QYjQXrxlkUzLl5m/OPU8//p93x/0/TF+GwVF3GyH38v32OyBVvGaQCxPd0Tqj5CtJ/T+mLsPAyAtiWO0VSA2vGeRSojta2472JfV/mLoMA8kLYljtFUgNrxnk2njlZrcvw0DZCDGs9gqkhtcMnMrtK72TvCCG1V6B1PCagZO5eaV3ykaIcfswI5BpvGbgdG5d6Z2RF8S4fZgRyDReMzBBdF7MTZefq5oLprrieiR5QRw3DzMCduA1A2QfK+xiTKwWCqSG1wxwalhhF6eM1V6B1PCaAbKHshEAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADAKyQsAADDKabkOAMkbGra0v+uEevsHVFpUqLkVU1QwyZPrsJCmfPt95tv5ArAPyYshmg8H1bCtU8HwQOxYwFeoNfWVqqsK5DAypCPffp/5dr4A7EXZyADNh4Na0XQo7o1fkkLhAa1oOqTmw8EcRYZ05NvvM9/OF4D9SF4cbmjYUsO2TlljfC96rGFbp4aGx2oBp8m332e+nS+A7CB5cbj9XSdG/cX6RZakYHhA+7tOZC8opC3ffp/5dr4AsoPkxeF6+8d/40+nHXIr336f+Xa+ALKD5MXhSosKM9oOuZVvv898O18A2ZGV5OXpp59WRUWFCgsLVV1drddff33ctq2trfJ4PKMe77zzTjZCdZy5FVMU8BVqvBtKPRq5a2NuxZRshoU05dvvM9/OF0B22J68bNmyRffdd58eeeQRtbe366qrrtLChQvV3d2d8HlHjhxRMBiMPS688EK7Q3WkgkkeramvlKRRHwDRr9fUV7JehiHy7feZb+cLIDtsT16efPJJ/cM//IP+8R//UZdcconWrVun8vJybdiwIeHzSktL5ff7Y4+CggK7Q3WsuqqANtw1W35f/NC631eoDXfNZp0Mw+Tb7zPfzheA/WxdpO7kyZN644039NBDD8Udr62t1d69exM+d9asWRoYGFBlZaX+9V//Vdddd92Y7QYHBzU4OBj7OhKJnHrgDlRXFdCCSj8rlLpEvv0+8+18AdjL1uTl+PHjGhoaUllZWdzxsrIyhUKhMZ8TCAT0zDPPqLq6WoODg/rP//xP3XDDDWptbdXVV189qn1jY6MaGhpsid9pCiZ5VHPB1FyHgQzJt99nvp0vAPtkZXsAjyf+ryvLskYdi5o5c6ZmzpwZ+7qmpkY9PT16/PHHx0xeVq9erVWrVsW+jkQiKi8vz1DkAADAaWyd81JSUqKCgoJRoyy9vb2jRmMSmTdvnt59990xv+f1elVcXBz3AAAA7mVr8nL66aerurpaLS0tccdbWlo0f/78pP+f9vZ2BQJM6gMAAFkoG61atUpLlizRnDlzVFNTo2eeeUbd3d1avny5pJGyz7Fjx7R582ZJ0rp16zRjxgxdeumlOnnypJqamrR161Zt3brV7lABAIABbE9ebr/9dvX19el73/uegsGgqqqqtGPHDk2fPl2SFAwG49Z8OXnypB544AEdO3ZMZ5xxhi699FJt375dixYtsjtUAABgAI9lWa7azjUSicjn8ykcDjP/BQBsMjRsces7MiqVz++s3G0EAHCP5sNBNWzrjNsxPOAr1Jr6ShYdRFawMSMAIGnNh4Na0XQoLnGRpFB4QCuaDqn5cDBHkSGfkLwAAJIyNGypYVunxpprED3WsK1TQ8Oumo0AByJ5AQAkZX/XiVEjLl9kSQqGB7S/60T2gkJeInkBACSlt3/8xCWddkC6SF4AAEkpLSqcuFEK7YB0kbwAAJIyt2KKAr5CjXdDtEcjdx3NrZiSzbCQh0heAABJKZjk0Zr6SkkalcBEv15TX8l6L7AdyQsAIGl1VQFtuGu2/L740pDfV6gNd81mnRdkBYvUAQBSUlcV0IJKPyvsImdIXgAAKSuY5FHNBVNzHUbGuHm7AzeeG8kLACCvuXm7A7eeG3NeAAB5y83bHbj53EheAAB5yc3bHbj53CSSFwBAnnLzdgduPjeJ5AUAkKfcvN2Bm89NInkBAOQpN2934OZzk0heAAB5ys3bHbj53CSSFwBAnnLzdgduPjeJ5AUAkMfcvN2Bm8/NY1mWmfdJjSMSicjn8ykcDqu4uDjX4QAADODGVWijTDm3VD6/WWEXAJD33LbdwRe58dwoGwEAAKOQvAAAAKOQvAAAAKOQvAAAAKOQvAAAAKNwtxHgYKbc4ggA2UTyAjhU8+GgGrZ1xu0MG/AVak19pdGLSwHAqaJsBDhQ8+GgVjQdGrWlfSg8oBVNh9R8OJijyAAg90heAIcZGrbUsK1TYy19HT3WsK1TQ8OuWhwbyKmhYUttR/v0SscxtR3t4/XlcJSNAIfZ33Vi1IjLF1mSguEB7e864bpVM4FcoERrHkZeAIfp7R8/cUmnHYDxUaI1E8kL4DClRYUTN0qhHYCxUaI1F8mLy1C3Nd/ciikK+Ao13g3RHo0Mac+tmJLNsADXSaVEC2dhzouLULd1h4JJHq2pr9SKpkPySHF/FUYTmjX1laz3ApwiSrTmYuTFJajbuktdVUAb7potvy++NOT3FWrDXbNJRoEMoERrrqwkL08//bQqKipUWFio6upqvf766wnb7969W9XV1SosLNT555+vjRs3ZiNMY1G3dZZMle7qqgL65YPX64Vl8/Rvd1yuF5bN0y8fvJ7EZQyUS5EOSrTmsr1stGXLFt133316+umn9dd//df693//dy1cuFCdnZ0677zzRrXv6urSokWLtGzZMjU1NelXv/qV7rnnHp1zzjm69dZb7Q7XSNxa6xyZLt0VTPLwO5sA5VKkixKtuTyWZdn6J8oVV1yh2bNna8OGDbFjl1xyiW6++WY1NjaOav/ggw/q1Vdf1dtvvx07tnz5cv3mN79RW1vbhD8vEonI5/MpHA6ruLg4MyfhcK90HNP//UXHhO3+7Y7LddPl59ofUJ6Klu6+/IKKvu1R7sk8+hyZQALsDKl8fts68nLy5Em98cYbeuihh+KO19bWau/evWM+p62tTbW1tXHHbrzxRm3atEmfffaZJk+ebFu8pqJum3sTle48GindLaj081dchtDnyJS6qoAWVPrZBNUgtiYvx48f19DQkMrKyuKOl5WVKRQKjfmcUCg0ZvvPP/9cx48fVyAQnwUPDg5qcHAw9nUkEslQ9OaI1m1D4YEx38g9GpnoSd3WPpTuso8+RyZRojVLVibsejzx2atlWaOOTdR+rOOS1NjYKJ/PF3uUl5dnIGKzROu2kkZNPKNumx3ccpl99DmQv2xNXkpKSlRQUDBqlKW3t3fU6EqU3+8fs/1pp52mqVNHZ8WrV69WOByOPXp6ejJ3Agbh1trconSXffQ5kL9sLRudfvrpqq6uVktLixYvXhw73tLSoptuumnM59TU1Gjbtm1xx1577TXNmTNnzPkuXq9XXq83s4Ebirpt7lC6yz76PD1DwxbvETCe7bdKr1q1SkuWLNGcOXNUU1OjZ555Rt3d3Vq+fLmkkZGTY8eOafPmzZJG7ixav369Vq1apWXLlqmtrU2bNm3SCy+8YHeorkDdNje45TL76PPUcVcN3ML2OS+333671q1bp+9973u6/PLLtWfPHu3YsUPTp0+XJAWDQXV3d8faV1RUaMeOHWptbdXll1+u73//+3rqqadY4wWOR+ku++jz5LEKN9zE9nVesi0f13mBszAsn31u6/NMn8/QsKUrH9s57t1Z0RLbLx+83uh+S5bbrhe3cMw6L0A+onSXfW7qcztKO9xW/meUztyBjRkBwCHsKu1wW/kISmfuQfICAA5g5war3FbOBrZuQ/ICAA6QSmknVeyebG//IvtIXgDAAews7bAKN6UztyF5AQAHsLu0k++3lVM6cxfuNgIAB8jGisH5vAo3KzK7CyMvAOAA2SrtRG8rv+nyc1VzwdS8SFwkSmduQ/ICAA6R76Udu9G/7sEKuwDgMKwAay/615lYYRcADOamFYOdiP41H2UjAABgFJIXAABgFJIXAABgFJIXAABgFJIXAABgFO42AgAYydRbnk2N20lIXgAAxmk+HFTDts64naIDvkKtqa909GJzpsbtNJSNAABGaT4c1IqmQ3EJgCSFwgNa0XRIzYeDOYosMVPjdiKSFwCAMYaGLTVs6xxzc8XosYZtnRoadtbi8abG7VQkLwYZGrbUdrRPr3QcU9vRPi5yAHlnf9eJUSMXX2RJCoYHtL/rRPaCSoKpcTsVc14MQZ0UAKTe/vETgHTaZYupcTsVIy8GoE4KACNKiwonbpRCu2wxNW6nInlxOOqkAPBncyumKOAr1Hg3Fns0Mio9t2JKNsOakKlxOxXJi8NRJwWAPyuY5NGa+kpJGpUIRL9eU1/puHVTTI3bqUheHI46KQDEq6sKaMNds+X3xZdY/L5CbbhrtmPnAZoatxMxYdfhqJMCwGh1VQEtqPQbt1KtqXE7DcmLw0XrpKHwwJjzXjwaydqpkwLINwWTPKq5YGquw0iZqXE7CWUjh6NOCgBAPJIXA1AnBQDgzygbGYI6KQAAI0heDEKdFAAAykYAAMAwJC8AAMAolI2ADBsatpibBAA2InkBMojdvwHAfpSNgAxh928AyA6SFyAD2P0bQD4YGrbUdrRPr3QcU9vRvpy9p9mavHz00UdasmSJfD6ffD6flixZoo8//jjhc+6++255PJ64x7x58+wMEzhl7P4NwO2aDwd15WM79c2f7tP//UWHvvnTfbrysZ05GVW2NXm588471dHRoebmZjU3N6ujo0NLliyZ8Hl1dXUKBoOxx44dO+wMEzhl7P4NwM2cVha3bcLu22+/rebmZu3bt09XXHGFJOmnP/2pampqdOTIEc2cOXPc53q9Xvn9frtCAzKO3b8BuNVEZXGPRsriCyr9Wbuz0raRl7a2Nvl8vljiIknz5s2Tz+fT3r17Ez63tbVVpaWluuiii7Rs2TL19vaO23ZwcFCRSCTuAWRbdPfv8V62Ho3cdcTu3wBM48SyuG3JSygUUmlp6ajjpaWlCoVC4z5v4cKFev7557Vz50498cQTOnDggK6//noNDg6O2b6xsTE2p8bn86m8vDxj5wAki92/AbiVE8viKScva9euHTWh9suPgwcPSpI8ntFv1JZljXk86vbbb9ff/M3fqKqqSvX19frv//5v/e53v9P27dvHbL969WqFw+HYo6enJ9VTAjKC3b8BuJETy+Ipz3lZuXKl7rjjjoRtZsyYoTfffFN/+MMfRn3vww8/VFlZWdI/LxAIaPr06Xr33XfH/L7X65XX6036/wPsxO7fANwmWhYPhQfGnPfi0cgfadksi6ecvJSUlKikpGTCdjU1NQqHw9q/f7/mzp0rSfr1r3+tcDis+fPnJ/3z+vr61NPTo0CAv1phBnb/BuAm0bL4iqZD8khxCUyuyuK2zXm55JJLVFdXp2XLlmnfvn3at2+fli1bpm984xtxdxpdfPHFeumllyRJn3zyiR544AG1tbXpvffeU2trq+rr61VSUqLFixfbFSoAAEjAaWVxW/c2ev7553XvvfeqtrZWkvS3f/u3Wr9+fVybI0eOKBwOS5IKCgr01ltvafPmzfr4448VCAR03XXXacuWLSoqKrIzVAAAkICTyuIey7JctV55JBKRz+dTOBxWcXFxrsMBAABJSOXzm72NAACAUUheAACAUWyd8wKMZ2jYckTdFIC78V7jTiQvyLrmw0E1bOuMW2464CvUmvpKFnIDkDG817gXZSNkldN2JgXgTrzXuBvJC7Jmop1JpZGdSYeGXXUDHIAs473G/UhekDVO3JkUgPvwXuN+JC/IGifuTArAfXivcT+SF2SNE3cmBeA+vNe4H8kLsia6M+l4Nyl6NHInQDZ3JgXgPrzXuB/JC7ImujOppFFvKrnamRSA+/Be434kL8gqp+1MCsCdeK9xNzZmTBKrNGYW/QkgG3ivMUcqn9+ssJsEVmnMvIJJHtVcMDXXYQBwOd5r3Imy0QRYpREAAGdh5CWBiVZp9GhklcYFlX6GIQ2T7lAyQ9AAkHskLwmkskojw5LmSLcMSPkQAJyBslECrNLoPumWASkfAoBzkLwkwCqN7pLuZm1s8gYAzkLykgCrNLpLupu1sclbbg0NW2o72qdXOo6p7WgfSSIA5rwkEl2lcUXTIXmkuL+8WaXRPOmWASkf5g7zjACMhZGXCbBKo3ukWwakfJgbzDMCMB5GXpJQVxXQgko/t8gaLloGDIUHxpy/4tFIUvrlMmC6z0P6WKYAQCKMvCQpukrjTZefq5oLpvKGaaB0N2tjk7fsY54RgERIXpBX0i0DUj7MLuYZAUiEshHyTrplQMqH2cM8IwCJkLwgL6W7WRubvGUH84wAJELZCIDjMM8IQCIkLwAciXlGAMZD2QiAYzHPCMBYSF4AOBrzjAB8GWUjAABgFJIXAABgFJIXAABgFJIXAABgFJIXAABgFO42QkYNDVvc1goAsBXJCzKm+XBQDds643YDDvgKtaa+kgXFAAAZY2vZ6NFHH9X8+fN15pln6uyzz07qOZZlae3atZo2bZrOOOMMXXvttfrtb39rZ5jIgObDQa1oOhSXuEhSKDygFU2H1Hw4mKPIAABuY2vycvLkSd12221asWJF0s/50Y9+pCeffFLr16/XgQMH5Pf7tWDBAvX399sYKU7F0LClhm2dY26gFz3WsK1TQ8NjtQAAIDW2Ji8NDQ26//779fWvfz2p9pZlad26dXrkkUd0yy23qKqqSs8995z++Mc/6uc//7mdoeIU7O86MWrE5YssScHwgPZ3ncheUAAA13LU3UZdXV0KhUKqra2NHfN6vbrmmmu0d+/eMZ8zODioSCQS90B29faPn7ik0w4AgEQclbyEQiFJUllZWdzxsrKy2Pe+rLGxUT6fL/YoLy+3PU7EKy0qnLhRCu0AAEgk5eRl7dq18ng8CR8HDx48paA8nvhbay3LGnUsavXq1QqHw7FHT0/PKf1spG5uxRQFfIUa74Zoj0buOppbMSWbYQEAXCrlW6VXrlypO+64I2GbGTNmpBWM3++XNDICEwj8+dba3t7eUaMxUV6vV16vN62fh8womOTRmvpKrWg6JI8UN3E3mtCsqa9kvRcAQEaknLyUlJSopKTEjlhUUVEhv9+vlpYWzZo1S9LIHUu7d+/WY489ZsvPRGbUVQW04a7Zo9Z58bPOCwAgw2xdpK67u1snTpxQd3e3hoaG1NHRIUn62te+prPOOkuSdPHFF6uxsVGLFy+Wx+PRfffdpx/84Ae68MILdeGFF+oHP/iBzjzzTN155512hooMqKsKaEGlnxV2AQC2sjV5+e53v6vnnnsu9nV0NGXXrl269tprJUlHjhxROByOtfnOd76jP/3pT7rnnnv00Ucf6YorrtBrr72moqIiO0NFhhRM8qjmgqm5DgMA4GIey7JctXJYJBKRz+dTOBxWcXFxrsMBAABJSOXz21G3SgMAAEyEjRlxSthFGrAfrzOkIh+uF5IXpI1dpAH78TpDKvLleqFshLSwizRgP15nSEU+XS8kL0gZu0gD9uN1hlTk2/VC8oKUsYs0YD9eZ0hFvl0vJC9IGbtIA/bjdYZU5Nv1QvKClLGLNGA/XmdIRb5dLyQvSBm7SAP243WGVOTb9ULygpRFd5GWNOqFwi7SQGbwOkMq8u16IXlBWqK7SPt98UOQfl+hNtw121XrCQC5wusMqcin64W9jXBK8mElRyDXeJ0hFaZeL6l8frPCLk4Ju0gD9uN1hlTkw/VC2QgAABiFkRcAp8zUYWoAZiJ5AXBK8mUjOADOQdkIQNryaSM4AM5B8gIgLfm2ERwA5yB5gVGGhi21He3TKx3H1Ha0jw/GHMq3jeAAOAdzXmAM5lY4S75tBAfAORh5gRGYW+E8+bYRHADnIHmB4zG3wpnybSM4AM5B8gLHY26FM+XbRnAAnIPkBY7H3ArnyqeN4AA4BxN24XjMrXC2uqqAFlT6WWEXQNaQvMDxonMrQuGBMee9eDTylz5zK3InHzaCA+AclI3geMytAAB8EckLjMDcCgBAFGUjGIO5FQAAieQFhmFuBQCAshEAADAKyQsAADAKZSMAgGsNDVvMk0uSSX1F8gIAcCV2ok+eaX1F2QgA4DrsRJ88E/uK5CVJQ8OW2o726ZWOY2o72uf6HYzz7XwBuAc70SfP1L6ibJQE04bTTlW+nS8Ad0llJ/p8X3rB1L6ydeTl0Ucf1fz583XmmWfq7LPPTuo5d999tzweT9xj3rx5doaZkInDaaci384XgPuwE33yTO0rW5OXkydP6rbbbtOKFStSel5dXZ2CwWDssWPHDpsiTMzU4bR05dv5AnCndHaiz9dSeTp95QS2lo0aGhokSc8++2xKz/N6vfL7/TZElBpTh9PSlW/nC8CdUt2JPp9L5an2lVM4csJua2urSktLddFFF2nZsmXq7e3NSRymDqelK9/OF4A7pbITfb6XylPpKydxXPKycOFCPf/889q5c6eeeOIJHThwQNdff70GBwfHbD84OKhIJBL3yBRTh9PSlW/nC8C9ktmJnlL5iGT6ymlSLhutXbs2Vg4az4EDBzRnzpy0Arr99ttj/66qqtKcOXM0ffp0bd++Xbfccsuo9o2NjRPGky5Th9PSlW/nC8DdJtqJnlL5n03UV06TcvKycuVK3XHHHQnbzJgxI914RgkEApo+fbrefffdMb+/evVqrVq1KvZ1JBJReXl5Rn52dDhtRdMheaS4D3QnD6elK9/OF4D7JdqJnlJ5vER95TQpJy8lJSUqKSmxI5Yx9fX1qaenR4HA2MNWXq9XXq/Xtp8fHU778mQuv0snc+Xb+QLIX5TKzWXr3Ubd3d06ceKEuru7NTQ0pI6ODknS1772NZ111lmSpIsvvliNjY1avHixPvnkE61du1a33nqrAoGA3nvvPT388MMqKSnR4sWL7Qw1IdOG005Vvp0vgPxEqdxctiYv3/3ud/Xcc8/Fvp41a5YkadeuXbr22mslSUeOHFE4HJYkFRQU6K233tLmzZv18ccfKxAI6LrrrtOWLVtUVFRkZ6gTMmk4LRPy7XwB5B9K5ebyWJblqmnUkUhEPp9P4XBYxcXFuQ4HAOBw+bzOi5Ok8vnN3kYAgLxGqdw8JC8AgLxHqdwsjlukDgAAIBGSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBSSFwAAYBTXrbAb3aopEonkOBIAAJCs6Od2Mlsuui556e/vlySVl5fnOBIAAJCq/v5++Xy+hG1ct6v08PCwPvjgAxUVFcnjcdamWpFIROXl5erp6WHH6zHQP4nRP+OjbxKjfxKjfxLLVv9YlqX+/n5NmzZNkyYlntXiupGXSZMm6atf/Wquw0iouLiYF0gC9E9i9M/46JvE6J/E6J/EstE/E424RDFhFwAAGIXkBQAAGIXkJYu8Xq/WrFkjr9eb61Acif5JjP4ZH32TGP2TGP2TmBP7x3UTdgEAgLsx8gIAAIxC8gIAAIxC8gIAAIxC8gIAAIxC8mKzRx99VPPnz9eZZ56ps88+O6nn3H333fJ4PHGPefPm2RtoDqTTN5Zlae3atZo2bZrOOOMMXXvttfrtb39rb6A58tFHH2nJkiXy+Xzy+XxasmSJPv7444TPcfO18/TTT6uiokKFhYWqrq7W66+/nrD97t27VV1drcLCQp1//vnauHFjliLNjVT6p7W1ddR14vF49M4772Qx4uzZs2eP6uvrNW3aNHk8Hr388ssTPidfrp9U+8Yp1w7Ji81Onjyp2267TStWrEjpeXV1dQoGg7HHjh07bIowd9Lpmx/96Ed68skntX79eh04cEB+v18LFiyI7WnlJnfeeac6OjrU3Nys5uZmdXR0aMmSJRM+z43XzpYtW3TffffpkUceUXt7u6666iotXLhQ3d3dY7bv6urSokWLdNVVV6m9vV0PP/yw7r33Xm3dujXLkWdHqv0TdeTIkbhr5cILL8xSxNn16aef6rLLLtP69euTap9P10+qfROV82vHQlb87Gc/s3w+X1Jtly5dat100022xuMkyfbN8PCw5ff7rR/+8IexYwMDA5bP57M2btxoY4TZ19nZaUmy9u3bFzvW1tZmSbLeeeedcZ/n1mtn7ty51vLly+OOXXzxxdZDDz00ZvvvfOc71sUXXxx37J/+6Z+sefPm2RZjLqXaP7t27bIkWR999FEWonMWSdZLL72UsE2+XT9RyfSNU64dRl4cqrW1VaWlpbrooou0bNky9fb25jqknOvq6lIoFFJtbW3smNfr1TXXXKO9e/fmMLLMa2trk8/n0xVXXBE7Nm/ePPl8vgnP1W3XzsmTJ/XGG2/E/d4lqba2dty+aGtrG9X+xhtv1MGDB/XZZ5/ZFmsupNM/UbNmzVIgENANN9ygXbt22RmmUfLp+klXrq8dkhcHWrhwoZ5//nnt3LlTTzzxhA4cOKDrr79eg4ODuQ4tp0KhkCSprKws7nhZWVnse24RCoVUWlo66nhpaWnCc3XjtXP8+HENDQ2l9HsPhUJjtv/88891/Phx22LNhXT6JxAI6JlnntHWrVv14osvaubMmbrhhhu0Z8+ebITsePl0/aTKKdeO63aVzoa1a9eqoaEhYZsDBw5ozpw5af3/t99+e+zfVVVVmjNnjqZPn67t27frlltuSev/zBa7+0aSPB5P3NeWZY065lTJ9o80+jylic/V5GtnIqn+3sdqP9Zxt0ilf2bOnKmZM2fGvq6pqVFPT48ef/xxXX311bbGaYp8u36S5ZRrh+QlDStXrtQdd9yRsM2MGTMy9vMCgYCmT5+ud999N2P/p13s7Bu/3y9p5K+iQCAQO97b2zvqrySnSrZ/3nzzTf3hD38Y9b0PP/wwpXM16doZT0lJiQoKCkaNIiT6vfv9/jHbn3baaZo6daptseZCOv0zlnnz5qmpqSnT4Rkpn66fTMjFtUPykoaSkhKVlJRk7ef19fWpp6cn7gPbqezsm4qKCvn9frW0tGjWrFmSRur9u3fv1mOPPWbLz8y0ZPunpqZG4XBY+/fv19y5cyVJv/71rxUOhzV//vykf55J1854Tj/9dFVXV6ulpUWLFy+OHW9padFNN9005nNqamq0bdu2uGOvvfaa5syZo8mTJ9sab7al0z9jaW9vN/o6yaR8un4yISfXTi5nC+eD999/32pvb7caGhqss846y2pvb7fa29ut/v7+WJuZM2daL774omVZltXf32/9y7/8i7V3716rq6vL2rVrl1VTU2Ode+65ViQSydVp2CLVvrEsy/rhD39o+Xw+68UXX7Teeust65vf/KYVCARc1zeWZVl1dXXWX/7lX1ptbW1WW1ub9fWvf936xje+EdcmX66dX/ziF9bkyZOtTZs2WZ2dndZ9991nfeUrX7Hee+89y7Is66GHHrKWLFkSa/+///u/1plnnmndf//9Vmdnp7Vp0yZr8uTJ1n/913/l6hRslWr//PjHP7Zeeukl63e/+511+PBh66GHHrIkWVu3bs3VKdiqv78/9v4iyXryySet9vZ26/3337csK7+vn1T7xinXDsmLzZYuXWpJGvXYtWtXrI0k62c/+5llWZb1xz/+0aqtrbXOOecca/LkydZ5551nLV261Oru7s7NCdgo1b6xrJHbpdesWWP5/X7L6/VaV199tfXWW29lP/gs6Ovrs771rW9ZRUVFVlFRkfWtb31r1O2J+XTt/OQnP7GmT59unX766dbs2bOt3bt3x763dOlS65prrolr39raas2aNcs6/fTTrRkzZlgbNmzIcsTZlUr/PPbYY9YFF1xgFRYWWn/xF39hXXnlldb27dtzEHV2RG/v/fJj6dKllmXl9/WTat845drxWNb/n4UEAABgAG6VBgAARiF5AQAARiF5AQAARiF5AQAARiF5AQAARiF5AQAARiF5AQAARiF5AQAARiF5AQAARiF5AQAARiF5AQAARiF5AQAARvl/v0BADepeoHYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = np.array(X, dtype=float)\n",
    "X[:,0] = (X[:,0] - np.mean(X[:,0])) / (np.std(X[:,0]))\n",
    "X[:,1] = (X[:,1] - np.mean(X[:,1])) / (np.std(X[:,1]))\n",
    "plt.scatter(X[:,0], X[:,1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "451a87b6-2819-4bb0-b2e2-bc790f0fba1d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-4.662936703425657e-17\n",
      "0.9999999999999999\n"
     ]
    }
   ],
   "source": [
    "print(np.mean(X[:,0]))\n",
    "print(np.std(X[:,0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bbd13ee-5ede-4d5f-99d6-c9279237c6e1",
   "metadata": {},
   "source": [
    "**（3）自定义封装**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c6e2a29-3c18-45e8-846f-88b15f6bf767",
   "metadata": {},
   "source": [
    "对测试数据的归一化：\n",
    "$$\\frac{(x_{test}-x_{train\\_mean})}{x_{train\\_std}}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b46de16-82a3-462d-8821-af4e771d9a40",
   "metadata": {},
   "source": [
    "> [MyMLTools/preprocessing.py](MyMLTools/preprocessing.py)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "8ebc19cf-9369-470f-895b-0df4a5e2cf94",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5.83416667 3.08666667 3.70833333 1.17      ]\n",
      "[0.81019502 0.44327067 1.76401924 0.75317107]\n"
     ]
    }
   ],
   "source": [
    "iris = datasets.load_iris()\n",
    "X = iris.data\n",
    "y = iris.target\n",
    "from MyMLTools.train_test_split import train_test_split\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,test_radio=0.2, seed=666)\n",
    "from MyMLTools.preprocessing import StandardScaler\n",
    "standardScaler = StandardScaler()\n",
    "standardScaler.fit(X_train)\n",
    "print(standardScaler.mean)\n",
    "print(standardScaler.scale)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "9a06f4be-9fed-4c53-81a2-31261440dffb",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_standard = standardScaler.transform(X_train)\n",
    "X_test_standard = standardScaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "26ab9f5f-cfad-43bb-bfa0-253d3ef63be0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      "0.3333333333333333\n"
     ]
    }
   ],
   "source": [
    "from MyMLTools.neighbors import KNeighborsClassifier\n",
    "knn = KNeighborsClassifier(k=3)\n",
    "knn.fit(X_train_standard, y_train)\n",
    "print(knn.score(X_test_standard,y_test))\n",
    "print(knn.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1578b82c-ac3e-4fb4-9728-8f2dc282594a",
   "metadata": {},
   "source": [
    "**（4）使用sklearn中的相关内容**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "defcd845-7ce1-4ff9-a204-e51fa71fdb27",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n",
      "0.3333333333333333\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,random_state=666)\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "standardScaler = StandardScaler()\n",
    "standardScaler.fit(X_train)\n",
    "X_train_standard = standardScaler.transform(X_train)\n",
    "X_test_standard = standardScaler.transform(X_test)\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "knn = KNeighborsClassifier(n_neighbors=3)\n",
    "knn.fit(X_train_standard, y_train)\n",
    "print(knn.score(X_test_standard,y_test))\n",
    "print(knn.score(X_test, y_test))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
